[MUSIC PLAYING]
BRIAN YU: All right.
Welcome back, everyone, to an introduction to artificial intelligence
with Python.
Last time we took a look at search problems
in particular, where we have AI agents that
are trying to solve some sort of problem by taking actions
in some sort of environment, whether that environment
is trying to take actions by playing moves in a game,
or whether those actions are something like trying to figure out where to make
turns in order to get driving directions from point A to point B.
This time we're going to turn our attention more generally
to just this idea of knowledge.
The idea that a lot of intelligence is based on knowledge, especially
if we think about human intelligence.
People know information.
We know facts about the world.
And using that information that we know, we're
able to draw conclusions-- reason about the information
that we know in order to figure out how to do something or figure out
some other piece of information that we conclude based on the information we
already have available to us.
What we'd like to focus on now is the ability to take this idea of knowledge
and being able to reason based on knowledge,
and apply those ideas to artificial intelligence.
In particular, we're going to be building what
are known as knowledge-based agents.
Agents that are able to reason and act by representing knowledge internally.
Somehow inside of our AI, they have some understanding
of what it means to know something.
And ideally, they have some algorithms, or some techniques
they can use based on that knowledge that they know in order
to figure out the solution to a problem, or figure out
some additional piece of information that can be helpful in some sense.
So what do we mean by reasoning based on knowledge
to be able to draw conclusions?
Well, let's look at a simple example drawn from the world of Harry Potter.
We take one sentence that we know to be true.
If it didn't rain, then Harry visited Hagrid today.
So one fact that we might know about the world.
And then we take another fact.
Harry visited Hagrid or Dumbledore today, but not both.
So it tells us something about the world.
That Harry either visited Hagrid but not Dumbledore,
or Harry visited Dumbledore but not Hagrid.
And now we have a third piece of information about the world
that Harry visited Dumbledore today.
So we now have three pieces of information now.
Three facts inside of a knowledge base, so to speak-- information that we know.
And now we, as humans, can try and reason about this,
and figure out based on this information, what
additional information can we begin to conclude?
And well, looking at these last two statements,
Harry either visited Hagrid or Dumbledore but not both,
and we know that Harry visited Dumbledore today.
Well, then it's pretty reasonable that we
could draw the conclusion that, you know what, Harry must not
have visited Hagrid today.
Because based on a combination of these two statements,
we can draw this inference, so to speak.
A conclusion that Harry did not visit Hagrid today.
But it turns out we can even do a little bit better than that-- get some more
information-- by taking a look at this first statement
and reasoning about that.
This first statement says, if it didn't rain, then Harry visited Hagrid today.
So what does that mean?
In all cases where it didn't rain, then we know that Harry visited Hagrid.
But if we also know now that Harry did not visit Hagrid,
then it tells us something about our initial premise
that we were thinking about.
In particular, it tells us that it did rain today.
Because we can reason if it didn't rain, that Harry would have visited Hagrid.
But we know for a fact that Harry did not visit Hagrid today.
So it's this kind of reasoning, the sort of logical reasoning
where we use logic based on the information
that we know in order to take information and reach conclusions.
That is going to be the focus of what we're going to be talking about today.
How can we make our artificial intelligence logical so
that they can perform the same kinds of deduction, the same kinds of reasoning
that we've been doing so far.
Of course, humans reason about logic generally in terms of human language.
That I just now was speaking in English, talking in English,
about these sentences and trying to reason through how
it is that they relate to one another.
We're going to need to be a little bit more formal when
we turn our attention to computers and being
able to encode this notion of logic, and truthhood and falsehood inside
of a machine.
So we're going to need to introduce a few more terms and a few symbols that
will help us reason through this idea of logic
inside of an artificial intelligence.
And we'll begin with the idea of a sentence.
Now, a sentence in a natural language like English
is just something that I'm saying, like what I'm saying right now.
In the context of AI though, a sentence is just an assertion about the world
in what we're going to call a knowledge representation language,
some way of representing knowledge inside of our computers.
And the way that we're going to spend most of today,
reasoning about knowledge, is through a type
of logic known as propositional logic.
There are a number of different types of logic, some of which we'll touch on.
But propositional logic is based on a logic of propositions, or just
statements about the world.
And so we begin in propositional logic with the notion
of propositional symbols.
We will have certain symbols that are oftentimes just letters,
something like P or Q or R, where each of those symbols
is going to represent some fact or sentence about the world.
So P, for example, might represent the fact that it is raining.
And so P is going to be a symbol that represents that idea.
And Q, for example, might represent Harry visited Hagrid today.
Each of these propositional symbols represents some sentence
or some fact about the world.
But in addition to just having individual facts about the world,
we want some way to connect these propositional symbols together
in order to reason more complexly about other facts that might exist inside
of the world in which we're reasoning.
So in order to do that, we'll need to introduce some additional symbols that
are known as logical connectives.
Now, there are a number of these logical connectives,
but five of the most important, and the ones we're going to focus on today,
are these five up here, each represented by a logical symbol.
Not is represented by this symbol here.
And is represented as sort of an upside-down V. Or
is represented by a V shape.
Implication-- and we'll talk about what that means in just a moment--
is represented by an arrow.
And biconditional-- again, we'll talk about what that means in a moment--
is represented by these double arrows.
But these five logical connectives are the main ones
we're going to be focusing on in terms of thinking about how
it is that a computer can reason about facts
and draw conclusions based on the facts that it knows.
But in order to get there we need to take a look
at each of these logical connectives and build up an understanding for what
it is that they actually mean.
So let's go ahead and begin with the not symbol.
So this not symbol here.
And what we're going to show for each of these logical connectives
is what we're going to call a truth table.
A table that demonstrates what this word not
means when we attach it to a propositional symbol or any sentence
inside of our logical language.
And so the truth table for not is shown right here.
If P-- some propositional symbol or some other sentence, even--
is false, then not P is true.
And if P is true, then not P is false.
So you can imagine placing this not symbol
in front of some sentence of propositional logic.
Just says the opposite of that.
So if, for example, P represented it is raining,
then not P would represent the idea that it is not raining.
And as you might expect, if P is false, meaning if the sentence it is raining
is false, well, then the sentence not P must be true.
The sentence that it is not raining is, therefore, true.
So not, you can imagine, just takes whatever is in P and it inverts it.
It turns false into true, and true into false.
Much analogously to what the English word not means.
Just taking whatever comes after it and inverting it to mean the opposite.
Next up, and also very English-like, is this idea
of and, represented by this upside-down V shape, or this point shape.
And as opposed to just taking a single argument the way not does--
we have P and we have not P--
and is going to combine two different sentences
and propositional logic together.
So I might have one sentence P and another sentence Q,
and I want to combine them together to say P and Q.
And the general logic for what P and Q means is it
means that both of its operands are true.
P is true, and also, Q is true.
And so here's what that truth table looks like.
This time we have two variables, P and Q.
And when we have two variables, each of which can be in two possible states,
true or false, that leads to two squared,
or four, possible combinations of truth and falsehood.
So we have P is false and Q is false.
We have P is false and Q is true.
P is true and Q is false.
And then P and Q both are true.
And those are the only four possibilities
for what P and Q could mean.
And in each of those situations, this third column here, P and Q,
is telling us a little bit about what it actually means for P and Q to be true.
And we see that the only case where P and Q is true is in this fourth row
here, where P happens to be true, Q also happens to be true.
And in all other situations, P and Q is going to evaluate to false.
So this, again, is much in line with what our intuition of and might mean.
If I say P and Q, I probably mean that I expect both P and Q to be true.
Next up, also potentially consistent with what we mean,
is this word or, represented by this V shape, sort
of an upside-down and symbol.
And or, as the name might suggest, is true if either of its arguments
are true.
As long as P is true or Q is true, then P or Q is going to be true.
Which means the only time that P or Q is false is if both of its operands
are false.
If P is false and Q is false, then P or Q is going to be false.
But in all other cases, if at least one of the operands is true--
maybe they're both true--
in which case P or Q is going to evaluate to true.
Now, this is mostly consistent with the way that most people might use the word
or, in the sense of speaking the word or in normal English.
Though there is sometimes when we might say or where
we mean P or Q, but not both.
Or we mean sort of it can only be one or the other.
It's important to note that this symbol here, this or, means P or Q or both.
That those are totally OK.
As long as either or both of them are true, then the or
is going to evaluate to be true as well.
It's only in the case where all of the operands
are false that P or Q ultimately evaluates to false as well.
In logic there's another symbol known as the exclusive or, which
encodes this idea of exclusivity of, like, one or the other, but not both.
But we're not going to be focusing on that today.
Whenever we talk about or, we're always talking about either or both,
in this case, as represented by this truth table here.
So that now is not, and an, and or.
And next step is what we might call implication, as denoted
by this arrow symbol.
So we have P and Q. And this sentence here
will generally read as P implies Q. And what
P implies Q means is that if P is true, then Q is also true.
So I might say something like, if it is raining, then I will be indoors.
Meaning it is raining implies I will be indoors
is the logical sentence that I'm saying there.
And the truth table for this can sometimes be a little bit tricky.
So obviously, if P is true and Q is true, then P implies Q, that's true.
That definitely makes sense.
And it should also stand to reason that when P is true and Q is false,
then P implies Q is false.
Because if I said to you, if it is raining, then I will be indoors,
and it is raining, but I'm not indoors, well, then it
would seem to be that my original statement was not true.
P implies Q means that if P is true, then Q also needs to be true.
And if it's not, well, then the statement is false.
Also worth noting, though, is what happens when P is false.
When P is false, the implication makes no claim at all.
If I say something like, if it is raining, then I will be indoors,
and it turns out it's not raining, then in that case,
I am not making any statement as to whether
or not I will be indoors or not.
P implies Q just means that if P is true, Q must be true.
But if P is not true, then we make no claim about
whether or not Q is true at all.
So in either case, if P is false, it doesn't matter
what Q is, whether it's false or true.
We're not making any claim about Q whatsoever.
We can still evaluate the implication to true.
The only way that the implication is ever false
is if our premise, P is true, but the conclusion that we're drawing,
Q, happens to be false.
So in that case, we would say P does not imply Q in that case.
Finally, the last connective that we'll discuss is this biconditional.
You can think of a biconditional as a condition that goes in both directions.
So originally, when I said something like, if it is raining,
then I will be indoors.
I didn't say what would happen if it wasn't raining.
Maybe I'll be indoors, maybe I'll be outdoors.
This biconditional you can read as an if and only if.
So I can say, I will be indoors if and only if it is raining.
Meaning if it is raining, then I will be indoors.
And if I am indoors, it's reasonable to conclude that it is also raining.
So this biconditional is only true when P and Q are the same.
So if P is true and Q is true, then this biconditional is also true--
P implies Q. But also the reverse is true.
Q also implies P. So if P and Q both happen to be false,
we would still say it's true.
But in any of these other two situations, this P if and only
if Q is going to ultimately evaluate to false.
So a lot of trues and falses going on there,
but these five basic logical connectives are
going to form the core of the language of propositional logic,
the language that we're going to use in order to describe ideas,
and the language that we're going to use in order to reason about those ideas
in order to draw conclusions.
So let's now take a look at some of the additional terms
that we'll need to know about in order to go
about trying to form this language of propositional logic,
and writing AI that's actually able to understand this sort of logic.
The next thing we're going to need is the notion
of what is actually true about the world.
We have a whole bunch of propositional symbols--
P and Q and R and maybe others.
But we need some way of knowing, like, what actually is true in the world.
Is P true or false, is Q true or false, so on and so forth.
And to do that, we'll introduce the notion of a model.
A model just assigns a truth value where a truth value is either true or false
to every propositional symbol.
In other words, it's creating what we might call a possible world.
So let me give an example.
If, for example, I have two propositional symbols,
P is it is raining, and Q is it is a Tuesday,
a model just takes each of these two symbols
and assigns a truth value to them, either true or false.
So here's a sample model.
In this model, in other words, in this possible world,
it is possible that P is true, meaning it is raining, and Q is false,
meaning it is not a Tuesday.
But there are other possible worlds or other models as well.
There is some model where both of these variables are true.
Some model where both of these variables are false.
In fact, if there are N variables that are propositional symbols like this,
that are either true or false, then the number of possible models
is two to the N, because each of these possible models--
possible variables within my model could be
set to either true or false, if I don't know any information about it.
So now that I have the symbols--
the symbols and the connectives that I'm going
to need in order to construct these parts of knowledge,
we need some way to represent that knowledge.
And to do so, we're going to allow our AI access
to what we'll call a knowledge base.
And a knowledge base is really just a set of sentences
that our AI knows to be true.
Some set of sentences in propositional logic
that are things that our AI knows about the world.
And so we might tell our AI some information, information
about a situation that it finds itself in,
or situation about a problem that it happens to be trying to solve.
And we would give that information to the AI,
that the AI would store inside of its knowledge base.
And what happens next is the AI would like
to use that information in the knowledge base
to be able to draw conclusions about the rest of the world.
And what do those conclusions look like?
Well, to understand those conclusions, we'll
need to introduce one more idea, one more symbol, and that
is the notion of entailment.
So this sentence here, with this double turnstile and these Greek letters--
this is the Greek letter alpha and the Greek letter beta--
and we read this as alpha entails beta.
And alpha and beta here are just sentences in propositional logic.
And what this means is that alpha entails beta means that in every model,
in other words, in every possible world in which sentence a is true--
or sentence alpha is true, then sentence beta is also true.
So if something entails something else, if alpha entails beta,
it means that if I know alpha to be true,
then beta must, therefore, also be true.
So if my alpha is something like, I know that it is a Tuesday in January,
then a reasonable beta might be something
like, I know that it is January.
Because in all worlds, where it is a Tuesday in January,
I know for sure that it must be January, just by definition.
This first statement, or sentence about the world,
entails the second statement.
And we can reasonably use deduction, based on that first sentence,
to figure out that the second sentence is, in fact, true as well.
And ultimately, it's this idea of entailment
that we're going to try and encode into our computer.
We want our AI agent to be able to figure out
what the possible entailments are.
We want our AI to be able to take these three sentences, sentences like,
if it didn't rain, Harry visited Hagrid.
That Harry visited Hagrid or Dumbledore but not both.
And that Harry visited Dumbledore.
And just using that information, we'd like our AI
to be able to infer, or figure out, that using these three sentences inside
of a knowledge base, we can draw some conclusions.
In particular, we can draw the conclusions here
that one, Harry did not visit Hagrid today.
And we can draw the entailment two, that it did, in fact, rain today.
And this process is known as inference.
And that's what we're going to be focusing
on today, this process of deriving new sentences from old ones.
That I give you these three sentences, you put them in the knowledge base
in, say, the AI, and the AI is able to use some sort of inference algorithm
to figure out that these two sentences must also be true.
And that is how we define inference.
So let's take a look at an inference example
to see how we might actually go about inferring things in a human sense,
before we take a more algorithmic approach to see how we could
encode this idea of inference in AI.
And we'll see there are a number of ways that we can actually achieve this.
So again, we'll deal with a couple of propositional symbols.
We'll deal with P, Q and R. P is it is a Tuesday.
Q is it is raining.
And R is Harry will go for a run.
Three propositional symbols that we are just defining to mean this.
We're not saying anything yet about whether they're true or false.
We're just defining what they are.
Now we'll give ourselves, or an AI, access to a knowledge base,
abbreviated to KB, to knowledge that we know about the world.
We know this statement.
All right, so let's try to parse it.
The parentheses here are just used for precedent,
so we can see what associates with what.
But you would read this as P and not Q implies R.
All right, so what does that mean?
Let's put it piece by piece.
P is it is a Tuesday.
Q is it is raining.
So not Q is it is not raining.
And implies R is Harry will go for a run.
So the way to read this entire sentence in human natural language at least,
is if it is a Tuesday and it is not raining, then Harry will go for a run.
So if it is a Tuesday and it is not raining, then Harry will go for a run.
And that is now inside of our knowledge base.
And let's now imagine that our knowledge base has two other pieces
of information as well.
It has information that P is true, that it is a Tuesday.
And we also have the information not Q, that it is not raining.
That this sentence Q, it is raining, happens to be false.
And those are the three sentences that we have access to.
P and not Q implies R, P and not Q.
Using that information, we should be able to draw some inferences.
P and not Q is only true if both P and not Q are true.
Well, all right, we know that P is true.
And we know that not Q is true.
So we know that this whole expression is true.
And the definition of implication is if this whole thing on the left is true,
then this thing on the right must also be true.
So if we know that P and not Q is true, then R must be true as well.
So the inference we should be able to draw from all of this
is that R is true, and we know that Harry will go for a run,
by taking this knowledge inside of our knowledge base and being able to reason
based on that idea.
And so this ultimately, is the beginning of what we might consider
to be some sort of inference algorithm.
Some process that we can use to try and figure out whether or not
we can draw some conclusion.
And ultimately, what these inference algorithms are going to answer
is the central question about entailment.
Given some query about the world, something
we're wondering about the world, and we'll call that query alpha,
the question we want to ask, using these inference algorithms,
is does it KB, our knowledge base, entail alpha?
In other words, using only the information
we know inside of our knowledge base, the knowledge that we have access to,
can we conclude that this sentence alpha is true?
And that's ultimately what we would like to do.
So how can we do that?
How can we go about writing an algorithm that can look at this knowledge base
and figure out whether or not this query alpha is actually true?
Well, it turns out there are a couple of different algorithms for doing so.
And one of the simplest, perhaps, is known as model checking.
Now remember, that a model is just some assignment
of all of the propositional symbols inside of our language to a truth
value, true or false.
And you can think of a model as a possible world.
That there are many possible worlds where different things
might be true or false.
And we can enumerate all of them.
And the model checking algorithm does exactly that.
So what does our model tracking algorithm do?
Well, if we wanted to determine if our knowledge
base entails some query alpha, then we are
going to enumerate all possible models.
In other words, consider all possible values of true and false
for our variables.
All possible states in which our world can be in.
And if in every model where our knowledge base is true,
alpha is also true, then we know that the knowledge base entails alpha.
So let's take a closer look at that sentence
and try and figure out what it actually means.
If we know that in every model, in other words, in every possible world,
no matter what assignment of true and false to variables you give,
if we know that whenever our knowledge is true--
what we know to be true is true--
that this query alpha is also true.
Well, then it stands to reason that as long as our knowledge base is true,
then alpha must also be true.
And so this is going to form the foundation of our model checking
algorithm.
We're going to enumerate all of the possible worlds,
and ask ourselves, whenever the knowledge base is true, is alpha true?
And if that's the case, then we know alpha to be true.
And otherwise, there is no entailment.
Our knowledge base does not entail alpha.
All right, so this is a little bit abstract.
But let's take a look at an example to try and put real propositional symbols
to this idea.
So again, we'll work with the same example.
P is it is a Tuesday.
Q is it is raining.
R is Harry will go for a run.
Our knowledge base contains these pieces of information.
P and not Q implies R. We also know P, it is a Tuesday.
And not Q, it is not raining.
And our query, our alpha in this case, the thing we want to ask
is R. We want to know, is it guaranteed?
Is it entailed that Harry will go for a run.
So the first step is to enumerate all of the possible models.
We have three propositional symbols here, P, Q,
and R, which means we have 2 to the third power, or 8 possible models.
All false.
False, false, true.
False, true, false.
False, true, true.
Et cetera.
Eight possible ways you could assign true and false to all of these models.
And we might ask in each one of them, is the knowledge base true?
Here are the set of things that we know.
In which of these worlds could this knowledge base possibly apply to?
In which world is this knowledge base true?
Well, in the knowledge base, for example, we
know P. Like, we know it is a Tuesday.
Which means we know that these four-- first four rows-- where P is false,
none of those are going to be true, are going to work
for this particular knowledge base.
Our knowledge base is not true in those worlds.
Likewise, we also know not Q. We know that it is not raining.
So any of these models where Q is true, like these two and these two
here, those aren't going to work either because we know that Q is not true.
And finally, we also know that P and not Q
implies R. Which means that when P is true--
where P is true here--
and Q is false--
Q is false in these two--
then R must be true.
And if ever P is true, Q is false, but R is also false,
well, that doesn't satisfy this implication here.
That implication does not hold true under those situations.
So we could say that for our knowledge base,
we can conclude under which of these possible worlds is our knowledge base
true, and under which of the possible worlds is our knowledge base false.
And it turns out there is only one possible world where
our knowledge base is actually true.
In some cases, there might be multiple possible worlds
where the knowledge base is true.
But in this case it just so happens that there's only one.
One possible world where we can definitively
say something about our knowledge base.
And in this case, we would look at the query.
The query of R. Is R true?
R is true.
And so as a result, we can draw that conclusion.
And so this is this idea of model checking.
Enumerate all the possible models, and look in those possible models
to see whether or not if our knowledge base is true,
is the query in question true as well.
So let's now take a look at how we might actually
go about writing this in a programming language like Python.
Take a look at some actual code that would
encode this notion of propositional symbols,
and logic, and these connectives, like and, and or, and not an implication,
and so forth, and see what that code might actually look like.
So I've written in advance a logic library.
It's more detailed than we need to worry about entirely today.
But the important thing is that we have one class
for every type of logical symbol, or connective, that we might have.
So we just have one class for logical symbols,
for example, where every symbol is going to represent and store
some name for that particular symbol.
And we also have a class for not, that takes an operand.
So we might say not one symbol to say something is not true,
or some other sentence is not true.
We have one for and, one for or, so on and so forth.
And I'll just demonstrate how this works.
And you can take a look at the actual logic.py later on.
But go ahead and call this file, Harry.py.
We're going to store information about this world of Harry Potter,
for example.
So I'll go ahead and import from my logic module.
I'll import everything.
And in this library, in order to create a symbol, you use capital S symbol.
And I'll create a symbol for rain to mean it is raining, for example.
And I'll create a symbol for Hagrid to mean
Harry visited Hagrid is what the symbol is going to mean.
So this symbol means it is raining.
This symbol means Harry visited Hagrid.
And I'll add another symbol called Dumbledore
for Harry visited Dumbledore.
Now I'd like to save these symbols so that I can use them later,
as I do some logical analysis.
So I'll go ahead and save each one of them inside of a variable.
So, like, rain, Hagrid, and Dumbledore, that you
could call the variables anything.
And now that I have these logical symbols,
I can use logical connectives to combine them together.
So for example, if I have a sentence like, and rain and Hagrid,
for example--
which is not necessarily true, but just for demonstration--
I can now try and print out sentence.formula,
which is a function I wrote that takes a sentence and propositional logic
and just prints it out so that we, the programmers,
can now see this in order to get an understanding for how it actually
works.
So if I run Python Harry.py, what we'll see
is this sentence and propositional logic, rain and Hagrid.
This is the logical representation of what
we have here in our Python program of saying
and, whose arguments are rain and Hagrid.
So we're saying rain and Hagrid by encoding that idea.
And this is quite common in Python object-oriented programming
where you have a number of different classes,
and you pass arguments into them in order to create a new
and object, for example, in order to represent this idea.
But now what I'd like to do is somehow encode the knowledge
that I have about the world, in order to solve
that problem from the beginning of class,
where we talked about trying to figure out who Harry visited,
and trying to figure out if it's raining or if it's not raining.
And so what knowledge do I have?
I'll go ahead and create a new variable called knowledge.
And what do I know?
Well, I know the very first sentence that we
talked about was the idea that if it is not raining, then
Harry will visit Hagrid.
All right.
How do I encode the idea that it is not raining?
Well, I can use not and then the rain symbol.
So here's me saying that it is not raining.
And now the implication is that if it is not raining, then Harry visited Hagrid.
So I'll wrap this inside of an implication
to say, if it is not raining, this first argument to the implication,
well, then Harry visited Hagrid.
So I'm saying implication, the premise is that it's not raining.
And if it is not raining, then Harry visited Hagrid.
And I can print out knowledge.formula to see the logical formula
equivalent of that same idea.
So I run Python of Harry.py, and this is the logical formula
that we see as a result, which is a text-based version of what
we were looking at before.
That if it is not raining, then that implies that Harry visited Hagrid.
But there was additional information that we had access to as well.
In this case, we had access to the fact that Harry
visited either Hagrid or Dumbledore.
So how do I encode that?
Well, this means that in my knowledge, I've
really got multiple pieces of knowledge going on.
I know one thing, and another thing, and another thing.
So I'll go ahead and wrap all of my knowledge inside of an and.
And I'll move things on a new line just for good measure.
But I know multiple things.
So I'm saying knowledge is an and of multiple different sentences.
I know multiple different sentences to be true.
One such sentence that I know to be true is this implication
that if it is not raining, then Harry visited Hagrid.
Another such sentence that I know to be true is or Hagrid Dumbledore.
In other words, so Hagrid or Dumbledore is true,
because I know that Harry visited Hagrid or Dumbledore.
But I know more than that, actually.
That initial sentence from before said that Harry visited
Hagrid or Dumbledore, but not both.
So now I want a sentence that'll encode the idea that Harry didn't
visit both Hagrid and Dumbledore.
Well, the notion of Harry visiting Hagrid and Dumbledore
would be represented like this.
And of Hagrid and Dumbledore.
And if that is not true, I want to say not that, then I'll just
wrap this whole thing inside of a not.
So now these three lines, line 8 says that if it is not raining, then
Harry visited Hagrid.
Line 9 says Harry visited Hagrid or Dumbledore.
And line 10 says Harry didn't visit both Hagrid and Dumbledore.
That it is not true that both the Hagrid symbol and the Dumbledore symbol
are true.
Only one of them can be true.
And finally, the last piece of information that I knew
was the fact that Harry visited Dumbledore.
So these now are the pieces of knowledge that I know.
One sentence, and another sentence, and another, and another.
And I can print out what I know, just to see it a little bit more visually.
And here now is a logical representation of the information
that my computer is now internally representing
using these various different Python objects.
And again, take a look at logic.py if you want to take a look at how exactly
it's implementing this.
But no need to worry too much about all of the details there.
We're here saying that if it is not raining, then Harry visited Hagrid.
We're saying that Hagrid or Dumbledore is true.
And we're saying it is not the case that Hagrid and Dumbledore is true.
That they're not both true.
And we also know that Dumbledore is true.
So this long, logical sentence represents our knowledge base.
It is the thing that we know.
And now what we like to do is we like to use model checking to ask a query.
To ask a question like, based on this information,
do I know whether or not it's raining?
And we, as humans, we're able to logic our way through it
and figure out that, all right, based on these sentences we can conclude this
and that to figure out that yes, it must have been raining.
But now we'd like for the computer to do that as well.
So let's take a look at the model checking algorithm that
is going to follow that same pattern that we drew out
in pseudocode a moment ago.
So I've defined a function here in logic.py
that you can take a look at called model check.
Model check takes two arguments, the knowledge that I already know
and the query.
And the idea is, in order to do model checking,
I need to enumerate all of the possible models.
And for each of the possible models, I need to ask myself,
is the knowledge base true, and is the query true?
So the first thing I need to do is somehow
enumerate all of the possible models.
Meaning for all possible symbols that exist,
I need to assign true and false to each one of them
and see whether or not it's still true.
And so here is the way we're going to do that.
We're going to start--
so I've defined another helper function internally
that we'll get to in just a moment.
But this function starts by getting all of the symbols,
in both the knowledge and the query, by figuring out
what symbols am I dealing with?
In this case, the symbols I'm dealing with
are rain, and Hagrid, and Dumbledore.
But there might be other symbols, depending on the problem.
And we'll take a look soon at some examples of situations
where ultimately, we're going to need some additional symbols in order
to represent the problem.
And then we're going to run this check all function, which
is a helper function that's basically going to recursively call itself,
checking every possible configuration of propositional symbols.
So we start out by looking at this check all function, and what do we do?
So if not symbols means if we finish assigning all of the symbols.
We've assigned every symbol of value.
So far we haven't done that, but if we ever do, then we check.
In this model, is the knowledge true?
That's what this line is saying.
If we evaluate the knowledge, propositional logic formula,
using the models assignment of truth values, is the knowledge true?
If the knowledge is true, then we should return true, only if the query is true.
Because if the knowledge is true, we want the query to be true as well,
in order for there to be entailment.
Otherwise, we don't know that there-- otherwise, there
won't be an entailment.
If there's ever a situation where what we know in our knowledge is true,
but the query, the thing we're asking, happens to be false.
So this line here is checking that same idea.
That in all worlds where the knowledge is true, the query must also be true.
Otherwise, we can just return true, because if the knowledge isn't true,
then we don't care.
This is equivalent to when we were enumerating
this table from a moment ago.
In all situations where the knowledge base wasn't true--
all of these seven rows here--
we didn't care whether or not our query was true or not.
We only care to check whether the query is true
when the knowledge base is actually true,
which was just this green highlighted row right there.
So that logic is encoded using that statement there.
And otherwise, if we haven't assigned symbols yet,
which we haven't seen anything yet, then the first thing we do
is pop one of the symbols.
I make a copy of the symbols first, just to save an existing copy.
But I pop one symbol off of the remaining symbols,
so that I just pick one symbol at random.
And I create one copy of the model where that symbol is true,
and I create a second copy of the model where that symbol is false.
So I now have two copies of the model.
One where the symbol is true, and one where the symbol is false.
And I need to make sure that this entailment holds
in both of those models.
So I recursively check all on the model where the statement is true,
and check all on the model where the statement is false.
So again, you can take a look at that function
to try to get a sense for how exactly this logic is working.
But in effect, what it's doing is recursively calling this check
all function again and again and again.
And on every level of the recursion we're saying,
let's pick a new symbol that we haven't yet assigned.
Assign it to true.
And assign it to false.
And then check to make sure that the entailment holds in both cases.
Because ultimately, I need to check every possible world.
I need to take every combination of symbols
and try every combination of true and false
in order to figure out whether the entailment relation actually holds.
So that function we've written for you.
But in order to use that function inside of Harry.py, what I'll write
is something like this.
I would like to model check, based on the knowledge,
and then I provide, as a second argument, what the query is.
What the thing I want to ask is.
And what I want to ask in this case is, is it raining?
So model check, again, takes two arguments.
The first argument is the information that I know.
This knowledge.
Which in this case, is this information that was given to me at the beginning.
And the second argument, rain, is encoding the idea of the query.
What am I asking?
I would like to ask, based on this knowledge,
do I know for sure that it is raining?
And I can try and print out the result of that.
And when I run this program, I see that the answer is true.
That based on this information, I can conclusively say that it is raining.
Because using this model checking algorithm,
we were able to check that in every world where this knowledge is true,
it is raining.
In other words, there's no world where this knowledge is true
and it is not raining.
So you can conclude that it is, in fact, raining.
And this sort of logic can be applied to a number
of different types of problems.
That if confronted with a problem where some sort of logical deduction
can be used in order to try to solve it, you
might try thinking about what propositional symbols
you might need in order to represent that information.
And what statements and propositional logic
you might use in order to encode that information which you know.
And this process of trying to take a problem
and figure out what propositional symbols to use in order
to encode that idea, or how to represent it logically
is known as knowledge engineering.
That software engineers and AI engineers will take a problem
and try and figure out how to distill it down into knowledge
that is representable by a computer.
And if we can take any general purpose problem, some problem that we
find in the human world, and turn it into a problem
that computer's know how to solve, as by using
any number of different variables, well, then, we
can take a computer that is able to do something
like model checking or some other inference algorithm,
and actually figure out how to solve that problem.
So now we'll take a look at two or three examples of knowledge engineering
and practice.
Of taking some problem and figuring out how we can apply logical symbols
and use logical formulas to be able to encode that idea.
And we'll start with a very popular board game in the US and the UK,
known as Clue.
Now, in the game of Clue, there's a number of different factors
that are going on.
But the basic premise of the game if you've never played it before,
is that there are a number of different people--
for now we'll just use three, Colonel Mustard, Professor Plum,
and Miss Scarlet.
There are a number of different rooms, like a ballroom, a kitchen,
and a library.
And there are a number of different weapons-- a knife, a revolver,
and a wrench.
And three of these-- one person, one room, and one weapon--
is the solution to the mystery--
the murderer and what room they were in, and what weapon they happen to use.
And what happens at the beginning of the game is that all these cards are
randomly shuffled together, and three of them-- one person, one room,
and one weapon--
are placed into a sealed envelope that we don't know.
And we would like to figure out, using some sort of logical process, what's
inside the envelope.
Which person, which room, and which weapon.
And we do so by looking at some, but not all, of these cards here.
By looking at these cards to try and figure out what might be going on.
And so this is a very popular game, but let's now
try and formalize it and see if we could train a computer
to be able to play this game by reasoning through it logically.
So in order to do this we'll begin by thinking
about what propositional symbols we're ultimately going to need.
Remember again, that propositional symbols are just
some symbol, some variable, that can be either true or false in the world.
And so in this case, the propositional symbols
are really just going to correspond to each of the possible things that
could be inside the envelope.
Mustard is a propositional symbol, that in this case
will just be true, if Colonel Mustard is inside the envelope,
if he is the murderer.
And false otherwise.
And likewise, for Plum for Professor Plum, and Scarlet for Miss Scarlet,
and likewise for each of the rooms, and for each of the weapons.
We have one propositional symbol for each of these ideas.
Then using those propositional symbols we
can begin to create logical sentences, create knowledge
that we know about the world.
So for example, we know that someone is the murderer.
That one of the three people is, in fact, the murderer.
And how would we encode that?
Well, we don't know for sure who the murderer is,
but we know it is one person, or the second person, or the third person.
So I could say something like this: Mustard, or Plum, or Scarlet.
And this piece of knowledge encodes that one of these three people
is the murderer.
We don't know which, but one of these three things must be true.
What other information do we know?
Well, we know that, for example, one of the rooms
must have been the room in the envelope.
That the crime was committed either in the ballroom, or the kitchen,
or the library.
Again, right now we don't know which, but this
is knowledge we know at the outset-- knowledge that one of these three
must be inside the envelope.
And likewise we can say the same thing about the weapon.
That it was either the knife, or the revolver, or the wrench.
That one of those weapons must have been the weapon of choice,
and therefore, the weapon in the envelope.
And then as the game progresses, the game play
works by people get various different cards.
And using those cards, you can deduce information.
That if someone gives you a card, for example,
I have the Professor Plum card in my hand,
then I know the Professor Plum card can't be inside the envelope.
I know that Professor Plum is not the criminal.
So I know a piece of information, like not Plum, for example.
I know that Professor Plum has to be false.
This propositional symbol is not true.
And sometimes I might not know for sure that a particular card is not
in the middle.
But sometimes, someone will make a guess,
and I'll know that one of three possibilities is not true.
Like someone will guess Colonel Mustard in the library with the revolver,
or something to that effect.
And in that case, a card might be revealed that I don't see.
But if it is a card, and it is either Colonel Mustard, or the revolver,
or the library, then I know that at least one of them
can't be in the middle.
So I know something like, it is either not Mustard, or it is not the library,
or it is not the revolver.
Now maybe multiple of these are not true, but I know that at least one
of Mustard, library and revolver must, in fact, be false.
And so this now is a propositional logic representation of this game of Clue.
A way of encoding the knowledge that we know inside this game using
propositional logic that a computer algorithm, something
like model checking that we saw a moment ago,
can actually look at and understand.
So let's now take a look at some code to see how this algorithm might actually
work in practice.
All right.
So I'm now going to open up a file called clue.py,
which I've started already.
And what we'll see here is I've defined a couple of things.
I've defined some symbols initially.
Notice I have a symbol for Colonel Mustard, a symbol for Professor Plum,
a symbol for Miss Scarlet, all of which I put inside
of this list of characters.
I have a symbol for ballroom, and kitchen,
and library inside of a list of rooms.
And then I have symbols for knife, and revolver, and wrench.
These are my weapons.
And so all of these characters and rooms and weapons altogether,
those are my symbols.
And now I also have this check knowledge function.
And what the check knowledge function does is it takes my knowledge,
and it's going to try and draw conclusions about what I know.
So for example, we'll loop over all of the possible symbols and we'll check.
Do I know that that symbol is true?
And a symbol is going to be something like Professor Plum, or the knife,
or the library.
And if I know that it is true, in other words,
I know that it must be the card in the envelope,
then I'm going to print out, using a function called
C print, which prints things in color.
I'm going to print out the word yes, and I'm going to print that in green,
just to make it very clear to us.
And if we're not sure that the symbol is true,
maybe I can check to see if I'm sure that the symbol is not true.
Like, if I know for sure that it is not Professor Plum, for example.
And I do that by running model check again.
This time checking if my knowledge is not the symbol.
If I know for sure that the symbol is not true.
And if I don't know for sure that the symbol is not true,
because I say elif not model check, meaning I'm not sure
that the symbol is false, well, then I'll
go ahead and print out maybe next to the symbol.
Because maybe the symbol is true.
Maybe it's not.
I don't actually know.
So what knowledge do I actually have?
Well, let's try and represent my knowledge now.
So my knowledge is, I know a couple of things so I'll put them in an and.
And I know that one of the three people must be the criminal.
So I know or Mustard, Plum, Scarlet.
This is my way of encoding that it is either Colonel Mustard, or Professor
Plum, or Miss Scarlet.
I know that it must have happened in one of the rooms.
So I know or ballroom, kitchen, library, for example.
And I know that one of the weapons must have been used as well.
So I know or knife, revolver, wrench.
So that might be my initial knowledge.
That I know that it must have been one of the people.
I know it must have been in one of the rooms.
And I know that it must have been one of the weapons.
And I can see what that knowledge looks like as a formula,
by printing out knowledge.formula.
So I'll run Python clue.py.
And here now is the information that I know in logical format.
I know that it is Colonel Mustard, or Professor Plum, or Miss Scarlet.
And I know that it is the ballroom, the kitchen, or the library.
And I know that it is the knife, the revolver, or the wrench.
But I don't know much more than that.
I can't really draw any firm conclusions.
And in fact, we can see that if I try and do--
let me go ahead and run my knowledge check function on my knowledge.
Now let's check is this function that I--
or check knowledge rather.
Is this function that I just wrote that looks over all of the symbols
and tries to see what conclusions I can actually draw about any of the symbols.
So I'll go ahead and run clue.py and see what it is that I know.
And it seems that I don't really know anything for sure.
I have all three people are maybes.
All three of the rooms are maybes.
All three of the weapons are maybes.
I don't really know anything for certain just yet.
But now let me try and add some additional information
and see if additional information, additional knowledge,
can help us to logically reason our way through this process.
And we are just going to provide the information.
Our AI is going to take care of doing the inference and figuring out
what conclusions it's able to draw.
So I start with some cards, and those cards tell me something.
So if I have the Colonel Mustard card, for example,
I know that the Mustard symbol must be false.
In other words, Mustard is not the one in the envelope.
It's not the criminal.
So I can say, knowledge supports something called--
every and in this library supports .add, which is a way of adding knowledge,
or adding an additional logical sentence to an and clause.
So I can say, knowledge.add, not Mustard.
I happen to know, because I have the Mustard card,
that Colonel Mustard is not the suspect.
And maybe I have a couple of other cards, too.
Maybe I also have a card for the kitchen,
so I know it's not the kitchen.
And maybe I have another card that says that it is not the revolver.
So I have three cards--
Colonel Mustard, the kitchen, and the revolver.
And I encode that into my AI this way, by saying it's not Colonel Mustard,
it's not the kitchen, and it's not the revolver, and I know those to be true.
So now when I rerun clue.py, we'll see that I've been
able to eliminate some possibilities.
Before I wasn't sure if it was the knife, or the revolver, or the wrench.
Knife was maybe, revolver was maybe, wrench was maybe.
Now I'm down to just the knife and the wrench.
Between those two, I don't know which one it is-- they're both maybes--
but I've been able to eliminate the revolver, which is one
that I know to be false because I have the revolver card.
And so additional information might be acquired over the course of this game,
and we would represent that just by adding knowledge
to our knowledge set, or knowledge base that we've been building here.
So if, for example, we additionally got the information
that someone made a guess.
Someone guessed, like, Miss Scarlet in the library with the wrench.
And we know that that card was revealed, which means that one of those three
cards-- either Miss Scarlet, or the library, or the wrench--
one of those at minimum, must not be inside of the envelope.
So I could add some knowledge.
Say knowledge.add, and I'm going to add an or clause, because I
don't know for sure which one it's not.
But I know one of them is not in the envelope.
So it's either not Scarlet or it's not the library.
And or supports multiple arguments.
I can say it's also or not the wrench.
So at least one of those--
Scarlet, library, and wrench-- at least one of those needs to be false.
I don't know which, though.
Maybe it's multiple, maybe it's just one, but at least one I know
needs to hold.
And so now if I rerun clue.py, I don't actually
have any additional information just yet.
Nothing I can say conclusively.
I still know that maybe it's Professor Plum, maybe it's Miss Scarlet.
I haven't eliminated any options.
But let's imagine that I get some more information.
That someone shows me the Professor Plum card, for example.
So I say, all right.
Let's go back here.
Knowledge.add, not Plum.
So I have the Professor Plum card.
I know that Professor Plum is not in the middle.
I rerun clue.py, and right now, I'm able to draw some conclusions.
Now I've been able to eliminate Professor Plum.
And the only person that could remaining be is Miss Scarlet.
So I know, yes, Miss Scarlet, this variable must be true.
And I've been able to infer that based on the information I already had.
Now, between the ballroom and the library, and the knife and the wrench,
for those two, I'm still not sure.
So let's add one more piece of information.
Let's say that I know that it's not the ballroom.
Someone has shown me the ballroom card, so I know it's not the ballroom.
Which means at this point, I should be able to conclude that it's the library.
Let's see.
I'll say knowledge.add, not the ballroom.
And we'll go ahead and run that.
And it turns out that after all of this, not only can
I conclude that I know that it's the library.
But I also know that the weapon was the knife.
And that might have been an inference that was a little bit trickier.
Something I wouldn't have realized immediately.
But the AI, via this model checking algorithm,
is able to draw that conclusion.
That we know for sure that it must be Miss
Scarlet in the library with the knife.
And how do we know that?
Well, we know it from this or clause up here.
That we know that it's either not Scarlet, or it's not the library,
or it's not the wrench.
And given that we know that it is Miss Scarlet,
and we know that it is the library, then the only remaining
option for the weapon is that it is not the wrench, which
means that it must be the knife.
So we, as humans, now can go back and reason through that,
even though it might not have been immediately clear.
And that's one of the advantages of using an AI or some sort of algorithm
in order to do this, is that the computer
can exhaust all of these possibilities and try and figure out
what the solution actually should be.
And so for that reason, it's often helpful to be
able to represent knowledge in this way.
Knowledge engineering, some situation, where
we can use a computer to be able to represent knowledge and draw
conclusions based on that knowledge.
And anytime we can translate something into propositional logic
symbols like this, type of approach can be useful.
So you might be familiar with logic puzzles, where
you have to puzzle your way through trying to figure something out.
This is what a classic logic puzzle might look like.
Something like Gildaroy, Minerva, Pomona, and Horace each
belong to a different one of the four houses--
Gryffindor, Hufflepuff, Ravenclaw, and Slytherin.
And then we have some information, that Gildaroy
belongs to Gryffindor or Ravenclaw.
Pomona does not belong in Slytherin.
And Minerva does belong to Gryffindor.
We have a couple of pieces of information.
And using that information, we need to be
able to draw some conclusions about which person
should be assigned to which house.
And again, we can use the exact same idea to try and implement this notion.
So we need some propositional symbols.
And in this case, the propositional symbols
are going to get a little more complex, although, we'll
see ways to make this a little bit cleaner later on.
But we'll need 16 propositional symbols.
One for each person and house.
So we need to say-- remember, every propositional symbol
is either true or false.
So Gildaroy Gryffindor is either true or false.
Either he's in Gryffindor or he is not.
Likewise, Gildaroy Hufflepuff also true or false.
Either it is true or it's false.
And that's true for every combination of person and house
that we could come up with.
We have some sort of propositional symbol for each one of those.
Using this type of knowledge, we can then
begin to think about what types of logical sentences
we can say about the puzzle.
That if we know--
before even think about the information we were given,
we can think about the premise of the problem.
That every person is assigned to a different house.
So what does that tell us?
Well, it tells us sentences like this.
It tells us, like, Pomona Slytherin implies not Pomona Hufflepuff.
Something like if Pomona is in Slytherin,
then we know that Pomona is not in Hufflepuff.
And we know this for all four people and for all combinations of houses.
That no matter what you person you pick, if they're in one house,
then they're not in some other house.
So I'll probably have a whole bunch of knowledge statements
that are of this four.
That if we know Pomona's in Slytherin, then
we know Pomona is not in Hufflepuff.
We were also given the information that each person is in a different house.
So I also have pieces of knowledge that look something like this.
Minerva Ravenclaw implies not Gildaroy Ravenclaw.
If they're all in different houses, then if Minerva is in Ravenclaw,
then we know that Gildaroy is not in Ravenclaw as well.
And I have a whole bunch of similar sentences
like this that are expressing that idea for other people and other houses
as well.
And so in addition to sentences of these form,
I also have the knowledge that was given to me.
Information like, Gildaroy was in Gryffindor or in Ravenclaw
that would be represented like this.
Gildaroy Gryffindor or Gildaroy Ravenclaw.
And then using these sorts of sentences, I
can begin to draw some conclusions about the world.
So let's see an example of this.
We'll go ahead and actually try and implement this logic puzzle
to see if we can figure out what the answer is.
I'll go ahead and open up puzzle.py where I've already
started to implement this sort of idea.
I've defined a list of people and a list of houses.
And I've, so far, created one symbol for every person and for every house.
That's what this double for loop is doing-- looping over all people,
looping over all houses, creating a new symbol for each of them.
And then I've added some information.
I know that every person belongs to a house,
so I've added the information for every person--
that person Gryffindor, or person Hufflepuff,
or person Ravenclaw, or person Slytherin.
That one of those four things must be true.
Every person belongs to a house.
What other information do I know?
I also know that only one house per person.
So no person belongs to multiple houses.
So how does this work?
Well, this is going to be true for all people.
So I'll loop over every person.
And then I need to loop over all different pairs of houses.
The idea is I want to encode the idea that if Minerva is in Gryffindor,
then Minerva can't be in Ravenclaw.
So I'll loop over all houses, h1.
And I'll loop over all houses again, h2.
And as long as they're different, h1 not equal to H2,
then I'll add to my knowledge base this piece of information.
That implication, in other words, an if/then, if the person is in h1,
then I know that they are not in house h2.
So these lines here are encoding the notion
that for every person, if they belong to house one, then
they are not in house two.
And the other piece of logic we need to encode
is the idea that every house can only have one person.
In other words, if Pomona is Hufflepuff, then nobody else
is allowed to be in Hufflepuff either.
And that's the same logic but sort of backwards.
I loop over all of the houses, and loop over all different pairs of people.
So I loop over people once, loop over people again.
And only do this when the people are different, p1 not equal to p2.
And I add the knowledge that if, as given by the implication,
if person one belongs to the house, then it
is not the case that person two belongs to the same house.
So here I'm just encoding the knowledge that
represents the problems constraints.
I know that everyone's in a different house.
I know that any person can only belong to one house.
And I can now take my knowledge and try and print out the information
that I happen to know.
So I'll go ahead and print out knowledge.formula,
just to see this in action.
And I'll go ahead and skip this for now, but we'll
come back to this in a second.
Let's print out the knowledge that I know by running Python puzzle.py.
It's a lot of information, a lot that I have
to scroll through, because there's 16 different variables all going on.
But the basic idea if we scroll up to the very top
is I see my initial information.
Gildaroy is either in Gryffindor, or Gildaroy is and Hufflepuff,
or Gildaroy is and Ravenclaw, or Gildaroy is in Slytherin.
And then way more information as well.
So this is quite messy.
More than we really want to be looking at.
And soon, too, we'll see ways of representing this a little bit more
nicely using logic.
But for now, we can just say these are the variables that we're dealing with.
And now we'd like to add some information.
So the information we're going to add is Gildaroy is in Gryffindor
or he is in Ravenclaw.
So that knowledge was given to us.
So I'll go ahead and say knowledge.add, and I
know that either or Gildaroy Gryffindor or Gildaroy Ravenclaw.
One of those two things must be true.
I also know that Pomona was not in Slytherin.
So I can say knowledge.add, not this symbol.
Not the Pomona Slytherin symbol.
And then I can add the knowledge that Minerva is in Gryffindor by adding
the symbol Minerva Gryffindor.
So those are the pieces of knowledge that I know.
And this loop here at the bottom just loops over all of my symbols,
checks to see if the knowledge entails that symbol by calling this model check
function again.
And if it does, if we know the symbol is true, we print out the symbol.
So now I can run Python puzzle.py, and Python
is going to solve this puzzle for me.
We're able to conclude that Gildaroy belongs to Ravenclaw,
Pomona belongs to Hufflepuff, Minerva to Gryffindor,
and Horace to Slytherin just by encoding this knowledge inside the computer--
although, it was quite tedious to do in this case--
and as a result, we were able to get the conclusion from that as well.
And you can imagine this being applied to many sorts of different deductive
situations.
So not only these situations where we're trying
to deal with Harry Potter characters in this puzzle.
But if you've ever played games like Mastermind
where you're trying to figure out which order different colors go in
and trying to make predictions about it, I could tell you, for example.
Let's play a simplified version of Mastermind where there are four
colors-- red, blue, green, and yellow--
and they're in some order, but I'm not telling you what order.
You just have to make a guess, and I'll tell you
of red, blue, green, and yellow, how many of the four
you got in the right position.
So a simplified version of this game.
You might make a guess, like red, blue, green, yellow.
And I would tell you something like, two of those four
are in the correct position, but the other two are not.
Then you could reasonably make a guess and say, all right, let's try this.
Blue, red, green, yellow.
Try switching two of them around.
And this time maybe I tell you, you know what, none of those
are in the correct position.
And the question then is, all right, what is
the correct order of these four colors?
And we, as humans, could begin to reason this through.
All right, well, if none of these were correct, but two of these were correct,
well, it must have been because I switched the red and the blue.
Which means red and blue here must be correct.
Which means green and yellow are probably not correct.
You can begin to do this sort of deductive reasoning.
We can also equivalently try and take this and encode it inside
of our computer as well.
And it's going to be very similar to the logic puzzle
that we just did a moment ago.
So I won't spend too much time on this code because it is fairly similar.
But again, we have a whole bunch of colors,
and four different positions in which those colors can be.
And then we have some additional knowledge.
And I encode all of that knowledge.
And you can take a look at this code on your own time.
But I just want to demonstrate that when we run this code,
run Python mastermind.py and run and see what we get,
we ultimately are able to compute red in the zero position,
blue in the one position, yellow in the two position,
and green in the three position as the ordering of those symbols.
Now, ultimately, what you might have noticed
is this process was taking quite a long time.
And, in fact, model checking is not a particularly efficient algorithm.
What I need to do in order to model check
is take all of my possible different variables
and enumerate all of the possibilities that they could be in.
If I have n variables, I have 2 to the n possible worlds
that I need to be looking through in order to perform this model checking
algorithm.
And this is probably not tractable, especially
as we start to get to much larger and larger sets of data
where you have many, many more variables that are at play.
Right here we only have a relatively small number of variables,
so this sort of approach can actually work.
But as the number of variables increases,
model checking becomes less and less good of a way of trying
to solve these sorts of problems.
So while it might have been OK for something like Mastermind
to conclude that this is, indeed, the correct sequence,
where all four are in the correct position, what we'd like to do
is come up with some better ways to be able to make inferences, rather
than just enumerate all of the possibilities.
And to do so, what we'll transition to next is the idea of inference rules.
Some sort of rules that we can apply to take knowledge that already exists
and translate it into new forms of knowledge.
And the general way we'll structure inference rule
is by having a horizontal line here.
Anything above the line is going to represent a premise, something
that we know to be true.
And then anything below the line will be the conclusion
that we can arrive at after we apply the logic,
or from the inference rule that we're going to demonstrate.
So we'll do some of these inference rules
by demonstrating them in English first, but then translating them
into the world of propositional logic so you
can see what those inference rules actually look like.
So for example, let's imagine that I have access
to two pieces of information.
I know, for example, that if it is raining,
then Harry is inside, for example.
And let's say I also know it is raining.
Then most of us could reasonably then look at this information
and conclude that, all right, Harry must be inside.
This inference rule is known as modus ponens,
and it's phrased more formally in logic as this.
If we know that alpha implies beta, in other words, if alpha,
then beta, and we also know that alpha is true,
then we should be able to conclude that beta is also true.
We can apply this inference rule to take these two pieces of information
and generate this new piece of information.
Notice that this is a totally different approach from the model checking
approach, where the approach was look at all of the possible worlds
and see what's true in each of these worlds.
Here, we're not dealing with any specific world.
We're just dealing with the knowledge that we know
and what conclusions we can arrive at based on that knowledge.
That I know that A implies B, and I know A, and the conclusion is B.
And this should seem like a relatively obvious rule.
But of course, if alpha than beta, and we know alpha,
then we should be able to conclude that beta is also true.
And that's going to be true for many, maybe even all of the inference rules
that we'll take a look at.
You should be able to look at them and say, yeah,
of course that's going to be true.
But it's putting these all together, figuring out
the right combination of inference rules that
can be applied that ultimately is going to allow us to generate
interesting knowledge inside of our AI.
So that's modus ponens, this application of implication.
That if we know alpha, and we know that alpha implies beta,
then we can conclude beta.
Let's take a look at another example.
Fairly straightforward.
Something like Harry is friends with Ron and Hermione.
Based on that information, we can reasonably
conclude Harry is friends with Hermione.
That must also be true.
And this inference rule is known as and elimination.
And what and elimination says, is that if we have a situation where
alpha and beta are both true--
I have information alpha and beta--
well, then just alpha is true.
Or likewise, just beta is true.
That if I know that both parts are true, then one of those parts
must also be true.
Again, something obvious from the point of view of human intuition.
But a computer needs to be told this kind of information.
To be able to apply the inference rule, we
need to tell the computer that this is an inference rule that you
can apply so the computer has access to it,
and is able to use it in order to translate information
from one form to another.
In addition to that, let's take a look at another example
of an inference rule.
Something like, it is not true that Harry did not pass the test.
Bit of a tricky sentence to parse so read it again.
It is not true, or it is false, that Harry did not pass the test.
Well, if it is false that Harry did not pass the test,
then the only reasonable conclusion is that Harry did pass the test.
And so this, instead of being and elimination,
is what we call double negation elimination.
That if we have two negatives inside of our premise, then
we can just remove them altogether.
They cancel each other out.
One turns true to false, and the other one turns false back into true.
Phrased a little bit more formally, we say
that if the premise is not not alpha, then the conclusion we can draw
is just alpha.
We can say that alpha is true.
We'll take a look at a couple more of these.
If I have it is raining, then Harry is inside, how do I reframe this?
Well, this one is a little bit trickier.
But if I know if it is raining, then Harry is inside,
then I conclude one of two things must be true.
Either it is not raining, or Harry is inside.
Now, this one's trickier, so let's think about it a little bit.
This first premise here, if it is raining, then Harry is inside,
is saying that if I know that it is raining, then Harry must be inside.
So what is the other possible case?
Well, if Harry is not inside, then I know that it must not be raining.
So one of those two situations must be true.
Either it's not raining, or it is raining, in which case Harry is inside.
So the conclusion I can draw is either it is not raining, or it is raining,
so therefore, Harry is inside.
So this is a way to translate if/then statements into or statements.
And this is known as implication elimination.
And this is similar to what we actually did in the beginning
when we were first looking at those very first sentences about Harry and Hagrid
and Dumbledore.
And phrased a little bit more formally, this
says that if I have the implication alpha
implies beta, that I can draw the conclusion that either not alpha
or beta.
Because there are only two possibilities.
Either alpha is true or alpha is not true.
So one of those possibilities is alpha is not true.
But if alpha is true, well, then we can draw the conclusion
that beta must be true.
So either alpha is not true, or alpha is true, in which case beta is also true.
So this is one way to turn an implication into just a statement about
or.
In addition to eliminating implications, we can also
eliminate biconditionals as well.
So let's take an English example.
Something like, it is raining if and only if Harry is inside.
And this if and only if really sounds like that biconditional,
that double arrow sign that we saw in propositional logic not too long ago.
And what does this actually mean if we were to translate this?
Well, this means that if it is raining, then Harry is inside.
And if Harry is inside, then it is raining.
The implication goes both ways.
And this is what we would call biconditional elimination.
That I can take a biconditional, A if and only if B,
and translate that into something like this.
A implies B, and B implies A. So many of these inference rules
are taking logic that uses certain symbols
and turning them into different symbols, taking an implication
and turning it into an or.
Or taking a biconditional and turning it into implication.
And another example of it would be something like this.
It is not true that both Harry and Ron passed the test.
Well, all right, how do we translate that?
What does that mean?
Well, if it is not true that both of them passed the test,
well, then the reasonable conclusion we might draw is that at least one of them
didn't pass the test.
So the conclusion is either Harry did not pass the test,
or Ron did not pass the test, or both.
This is not an exclusive or.
But if it is true that it is not true, that both Harry and Ron passed
the test, well, then either Harry didn't pass the test
or Ron didn't pass the test.
And this type of law is one of De Morgan's laws.
Quite famous in logic, where the idea is that we can turn an and into an or.
We can take this and, that both Harry and Ron passed the test,
and turn it into an or by moving the nots around.
So if it is not true that Harry and Ron passed the test,
well, then either Harry did not pass the test,
or Ron did not pass the test either.
And the way we frame that more formally using logic is to say this.
If it is not true that alpha and beta, well, then either not
alpha or not beta.
The way I like to think about this is that if you
have a negation in front of an and expression,
you move the negation inwards, so to speak.
Moving the negation into each of these individual sentences,
and then flip the and into an or.
So the negation moves inwards, and the and flips into an or.
So I go from not A and B, to not A or not B.
And there's actually a reverse of De Morgan's law
that goes in the other direction for something like this.
If I say, it is not true that Harry or Ron passed the test,
meaning neither of them passed the test, well, then the conclusion I can draw
is that Harry did not pass the test, and Ron did not pass the test.
So in this case, instead of turning an and into an or, we're turning an
or into an and.
But the idea is the same.
And this, again, is another example of De Morgan's laws.
And the way that works is that if I have not A or B this time,
the same logic's going to apply.
I'm going to move the negation inwards, and I'm
going to flip, this time, flip the or into an and.
So if not A or B, meaning it is not true that A or B, or alpha or beta,
then I can say not alpha and not beta.
Moving the negation inwards in order to make that conclusion.
So those are De Morgan's laws.
And a couple of other inference rules that are worth just taking a look at,
one is the distributive law that works this way.
So if I have alpha and beta or gamma, well, then much in the same way
that you can use, in math, use distributive laws
to distribute operands, like addition and multiplication,
I can do a similar thing here.
Where I can say if alpha and beta or gamma,
then I can say something like, alpha and beta, or alpha and gamma.
That I've been able to distribute this and sign throughout this expression.
So this is an example of the distributive property
or the distributive law, as applied to logic in much the same way
that you would distribute like a multiplication over the addition
of something, for example.
This works the other way, too.
So if, for example, I have alpha or beta and gamma,
I can distribute the or throughout the expression.
I can say alpha or beta, and alpha or gamma.
So the distributive law works in that way, too.
And it's helpful if I want to take an or and move it into the expression.
And we'll see an example soon of why it is that we might actually
care to do something like that.
All right.
So now we've seen a lot of different inference rules.
And the question now is how can we use those inference rules to actually
try and draw some conclusions?
To actually try and prove something about entailment,
proving that given some initial knowledge base,
we would like to find some way to prove that a query is true.
Well, one way to think about it is actually
to think back to what we talked about last time,
when we talked about search problems.
Recall again that search problems have some sort of initial state.
They have actions that you can take from one state to another,
as defined by a transition model that tells you how to get from one state
to another.
We talked about testing to see if you had a goal.
And then some path cost function to see how many steps did you have to take,
or how costly was the solution that you found.
Now that we have these inference rules that
take some set of sentences and propositional logic
and get us some new set of sentences in propositional logic,
we can actually treat those sentences, or those sets of sentences,
as states inside of a search problem.
So if we want to prove that some query is true,
prove that some logical theorem is true, we
can treat theorem proving as a form of a search problem.
I can say that we begin in some initial state,
where that initial state is the knowledge base that I begin with.
The set of all of the sentences that I know to be true.
What actions are available to me?
Well, the actions are any of the inference rules
that I can apply at any given time.
The transition model just tells me after I apply the inference rule,
here is the new set of all of the knowledge that I have,
which will be the old set of knowledge, plus some additional inference
that I've been able to draw, much as in the same way we
saw what we got when we applied those inference rules
and got some sort of conclusion.
That conclusion gets added to our knowledge base,
and our transition model will encode that.
What is the goal test?
Well, our goal test is, you know, checking
to see if we have proved the statement we're trying to prove.
If the thing we're trying to prove is inside of our knowledge base.
And the path cost function, the thing we're trying to minimize,
is maybe the number of inference rules that we needed to use.
The number of steps, so to speak, inside of our proof.
And so here we've been able to apply the same types of ideas
that we saw last time with search problems,
to something like trying to prove something about knowledge
by taking our knowledge and framing it in terms that we can understand
as a search problem, with an initial state,
with actions, with a transition model.
So this shows a couple of things.
One being how versatile search problems are.
That they can be the same types of algorithms that we use to solve a maze,
or figure out how to get from point A to point B. Inside of driving directions,
for example, can also be used as a theorem proofing method.
Of taking some sort of starting knowledge base
and trying to prove something about that knowledge.
So this, yet again, is a second way, in addition to model checking,
to try and prove that certain statements are true.
But it turns out there's yet another way that we can try and apply inference,
and we'll talk about this now, which is not the only way,
but certainly one of the most common.
Which is known as resolution.
And resolution is based on another inference rule
that we'll take a look at now.
Quite a powerful inference rule that will
let us prove anything that can be proven about a knowledge base.
And it's based on this basic idea.
Let's say I know that either Ron is in the Great Hall,
or Hermione is in the library.
And let's say I also know that Ron is not in the Great Hall.
Based on those two pieces of information, what can I conclude?
Well, I could pretty reasonably conclude that Hermione must be in the library.
How do I know that?
Well, it's because these two statements, these two, what we'll call,
complementary literals-- literals that complement each other,
they are opposites of each other--
seem to conflict with each other.
This sentence tells us that either Ron is in the Great Hall
or Hermione is in the library.
So if we know that Ron is not in the Great Hall,
that conflicts with this one, which means Hermione must be in the library.
And this we can frame as a more general rule, known as the unit resolution
rule.
A rule that says that if we have P or Q, and we also know not P, well, then
from that we can reasonably conclude Q.
That if P or Q are true, and we know that P is not true,
the only possibility is for Q to then be true.
And this, it turns out, is quite a powerful inference rule
in terms of what it can do, in part because we can quickly
start to generalize this rule.
This Q right here doesn't need to just be a single propositional symbol.
It could be multiple, all chained together in a single clause,
as we'll call it.
So if I had something like P or Q1, or Q2, or Q3, so on and so forth,
up until Qn, so I had n, different, other variables,
and I have not P, well, then what happens when these two complement
each other is that these two clauses resolve, so to speak,
to produce a new clause that is just Q1 or Q2, all the way up to Qn.
And in an or, the order of the arguments in the or doesn't actually matter.
The P doesn't need to be the first thing.
It could've been in the middle.
But the idea here is that if I have P in one clause,
and not P and the other clause, well, then
I know that one of these remaining things must be true.
I've resolved them in order to produce a new clause.
But it turns out we can generalize this idea even further, in fact,
and display even more power that we can have with this resolution rule.
So let's take another example.
Let's say, for instance, that I know the same piece of information,
that either Ron is in the Great Hall or Hermione is in the library.
And the second piece of information I know
is that Ron is not in the Great Hall or Harry is sleeping.
So it's not just a single piece of information.
I have two different clauses, and we'll define clauses more precisely
in just a moment.
What do I know here?
Well, again, for any propositional symbol, like Ron is in the Great Hall,
there are only two possibilities.
Either Ron is in the Great Hall, in which case, based on resolution,
we know that Harry must be sleeping.
Or Ron is not in the Great Hall, in which case
we know, based on the same rule, that Hermione must be in the library.
Based on those two things in combination,
I can say, based on these two premises, that I
can conclude that either Hermione is in the library or Harry is sleeping.
So again, because these two conflict with each other,
I know that one of these two must be true.
And you can take a closer look and try and reason through that logic.
Make sure you convince yourself that you believe this conclusion.
Stated more generally, we can name this resolution rule
by saying that if we know P or Q is true,
and we also know that not P or R is true,
we resolve these two clauses together to get a new clause, Q
or R. That either Q or R must be true.
And again, much as in the last case, Q and R
don't need to just be single propositional symbols.
It could be multiple symbols.
So if I had a rule that had a P or Q1 or Q2 or Q3, so on and so forth,
up until Qn, where n is just some number.
And likewise, I had not P or R1 or F2, so on and so forth, up until Rm, m,
where m, again, is just some other number,
I can resolve these two clauses together to get one of these must be true.
Q1 or Q2, up until Qn.
Or R1 or R2, up until Rm.
And this is just a generalization of that same rule we saw before.
Each of these things here we're going to call a clause.
Where a clause is formally defined as a disjunction of literals.
Where a disjunction means it's a bunch of things that are connected with or.
Disjunction means things connected with or.
Conjunction, meanwhile, is things connected with and.
And a literal is either a propositional symbol or the opposite
of a propositional symbol.
So it's something like P or Q, or not P or not Q,
those are all propositional symbols, or not of the propositional symbols,
and we call those literals.
So a clause is just something like this.
P or Q or R, for example.
Meanwhile, what this gives us an ability to do
is it gives us an ability to turn logic, any logical sentence,
into something called conjunctive normal form.
A conjunctive normal form sentence is a logical sentence
that is a conjunction of clauses.
Recall again, conjunction means things are connected to one another using and.
And so a conjunction of clauses means that it
is an and of individual clauses, each of which has in it.
So something like this.
A or B or C, and D or not E, and F or G. Everything in parentheses
is one clause.
All of the clauses are connected to each other using an and,
and everything in the clause is separated using an or.
And this is just a standard form that we can translate a logical sentence
into that just makes it easy to work with and easy to manipulate.
And it turns out that we can take any sentence in logic
and turn it into conjunctive normal form,
just by applying some inference rules and transformations to it.
So we'll take a look at how we can actually do that.
So what is the process for taking a logical formula
and converting it into injunctive normal form, otherwise known as CNF?
Well, the process looks a little something like this.
We need to take all of the symbols that are not
part of conjunctive normal form-- the biconditionals, and the implications,
and so forth.
And turn them into something that is more closely
like conjunctive normal form.
So the first step will be to eliminate biconditionals.
Those if and only if double arrows.
And we know how to eliminate biconditionals
because we saw there was an inference rule to do just that.
Anytime I have an expression, like alpha, if and only if beta,
I can turn that into alpha implies beta, and beta implies alpha, based
on that inference rule we saw before.
Likewise, in addition to eliminating biconditionals,
I can eliminate implications as well.
The if/then arrows.
And I can do that using the same inference rule we saw before, too.
Taking alpha implies beta, and turning that
into not alpha or beta, because that is logically
equivalent to this first thing here.
Then we can move nots inwards, because we don't want nots
on the outsides of our expressions.
Conjunctive normal form requires that its just clause and clause
and clause and clause.
Any nots need to be immediately next to propositional symbols.
But we can move those nots around using De Morgan's laws.
By taking something like, not A and B, and turn it into not A or not B,
for example, using de Morgan's laws to manipulate that.
And after that, all we'll be left with are ands and ors,
and those are easy to deal with.
We can use the distributive law to distribute the ors so that the ors end
up on the inside of the expression, so to speak,
and the ands end up on the outside.
So this is the general pattern for how we'll take a formula
and convert it into conjunctive normal form.
And let's now take a look at an example of how
we would do this, and explore then, why it is that we
would want to do something like this.
Here's how we can do it.
Let's take this formula, for example.
P or Q implies R, and I'd like to convert this
into conjunctive normal form, where it's all ands of clauses,
and every clause is a disjunctive clause.
It's or is together.
So what's the first thing I need to do?
Well, this is an implication.
So let me go ahead and remove that implication.
Using the implication inference rule, I can turn P or Q into--
P or Q implies R, into not P or Q or R. So that's the first step.
I've gotten rid of the implication.
And next, I can get rid of the not on the outside of this expression, too.
I can move the nots inwards so they're closer to the literals themselves
by using De Morgan's laws.
And De Morgan's law says that not P or Q is equivalent
to not P and not Q. Again here, just applying the inference rules
that we've already seen in order to translate these statements.
And now I have two things that are separated by an
or, where this thing on the inside is an and.
What I'd really like is to move the or so the ors are on the inside,
because conjunctive normal form means I need clause and clause and clause
and clause.
And so to do that, I can use the distributive law.
If I have not P and not Q or R, I can distribute the or R to both of these
to get not P or R, and not Q or R using the distributive law.
And this now, here at the bottom, is in conjunctive normal form.
It is a conjunction, an and, of disjunctions,
of clauses that just are separated by ors.
So this process can be used by any formula to take a logical sentence
and turn it into this conjuncture normal form, where
I have clause and clause and clause and clause and clause, and so on.
So why is this helpful?
Why do we even care about taking all these sentences
and converting them into this form?
It's because once they're in this form where we have these clauses,
these clauses are the inputs to the resolution, inference rule that we
saw a moment ago.
That if I have two clauses where there's something that conflicts,
or something complementary between those two clauses,
I can resolve them to get a new clause, to draw a new conclusion.
And we call this process inference by resolution.
Using the resolution rule to draw some sort of inference.
And it's based on the same idea, that if I have P or Q, this clause,
and I have not P or R, that I can resolve these two clauses together
to get Q or R as the resulting clause.
A new piece of information that I didn't have before.
Now, a couple of key points that are worth noting about this before we
talk about the actual algorithm.
One thing is that, let's imagine we have P or Q or S,
and I also have not P or R or S. The resolution rule says
that because this P conflicts with this not P,
we would resolve to put everything else together, to get Q or S or R or S.
But it turns out that this double S is redundant-- or S here and or S there.
It doesn't change the meaning of the sentence.
So in resolution, when we do this resolution process,
we usually also do a process known as factoring,
where we take any duplicate variables that show up and just eliminate them.
So Q or S or R or S just becomes Q or R or S. The S only needs to appear once.
No need to include it multiple times.
Now, one final question worth considering
is what happens if I try to resolve P and not P together?
If I know that P is true, and I know that not P is true,
well, resolution says I can merge these clauses together and look
at everything else.
Well, in this case, there is nothing else.
So I'm left with what we might call the empty clause.
I'm left with nothing.
And the empty clause is always false.
The empty clause is equivalent to just being false.
And that's pretty reasonable.
Because it's impossible for both P and not P to both hold at the same time.
P is either true or it's not true.
Which means that if P is true, then this must be false.
And if this is true, then this must be false.
There's no way for both of these to hold at the same time.
So if ever I try and resolve these two, it's a contradiction,
and I'll end up getting this empty clause, where the empty clause
I can call equivalent to false.
And this idea that if I resolve these two contradictory terms I
get the empty clause, this is the basis for our inference by resolution
algorithm.
Here is how we're going to perform inference
by resolution at a very high level.
We want to prove that our knowledge base entails some query alpha.
That based on the knowledge we have, we can prove conclusively
that alpha is going to be true.
How are we going to do that?
Well, in order to do that, we're going to try
to prove that if we know the knowledge and not alpha, that that
would be a contradiction.
And this is a common technique in computer science more generally.
This idea of proving something by contradiction.
If I want to prove that something is true,
I can do so by first assuming that it is false,
and showing that it would be contradictory.
Showing that it leads to some contradiction.
And if the thing I'm trying to prove, if when I assume it's false
leads to a contradiction, then it must be true.
And that's the logical approach, or the idea, behind a proof by contradiction.
And that's what we're going to do here.
We want to prove that this query alpha is true,
so we're going to assume that it's not true.
We're going to assume not alpha.
And we're going to try and prove that it's a contradiction.
If we do get a contradiction, well, then we
know that our knowledge entails the query alpha.
If we don't get a contradiction, there is no entail.
This is this idea of a proof by contradiction
of assuming the opposite of what you're trying to prove,
and if you can demonstrate that that's a contradiction, then what you're proving
must be true.
But more formally, how do we actually do this?
How do we check that knowledge base and not alpha
is going to lead to a contradiction?
Well, here is where resolution comes into play.
To determine if our knowledge base entails some query alpha,
we're going to convert knowledge base and not alpha to conjunction
normal form.
That form where we have a whole bunch of clauses that are all anded together.
And when we have these individual clauses,
now we can keep checking to see if we can use resolution
to produce a new clause.
We can take any pair of clauses and check.
Is there some literal that is the opposite of each other,
or complementary to each other, in both of them?
For example, I have a P in one clause, and a not P in another clause.
Or an R in one clause, and a not R in another clause.
If ever I have that situation, where once I
convert to conjunctive normal form and I have a whole bunch of clauses,
I see two clauses that I can resolve to produce a new clause, then I'll do so.
This process occurs in a loop.
I'm going to keep checking to see if I can use resolution
to produce a new clause, and keep using those new clauses to try to generate
more new clauses after that.
Now, it just so may happen that eventually, we
may produce the empty clause.
The clause we were talking about before.
If I resolve P and not P together, that produces the empty clause.
And the empty clause we know to be false.
Because we know that there's no way for both P and not
P to both simultaneously be true.
So if ever we produce the empty clause, then we have a contradiction.
And if we have a contradiction, that's exactly what
we were trying to do in a proof by contradiction.
If we have a contradiction, then we know that our knowledge base
must entail this query alpha.
We know that alpha must be true.
And it turns out-- and we won't go into the proof here--
but you can show that, otherwise, if you don't produce the empty clause,
then there is no entailment.
If we run into a situation where there are no more new clauses to add,
we've done all the resolution that we can do,
and yet we still haven't produced the empty clause,
then there is no entailment in this case.
And this now is the resolution algorithm.
And it's very abstract looking, especially
this idea of what does that even mean to have the empty clause.
So let's take a look at an example.
Actually try and prove some entailment by using this inference
by resolution process.
So here's our question.
We have this knowledge base.
Here is the knowledge that we know.
A or B, and not B or C, and not C. And we want to know if all of this
entails A.
So this is our knowledge base here, this whole log thing.
And our query alpha is just this propositional symbol A.
So what do we do?
Well, first we want to prove by contradiction.
So we want to first assume that A is false,
and see if that leads to some sort of contradiction.
So here is what we're going to start with.
A or B, and not B or C, and not C. This is our knowledge base.
And we're going to assume not A. We're going
to assume that the thing we're trying to prove is, in fact, false.
And so this is now in conjunctive normal form,
and I have four different clauses.
I have A or B. I have not B or C. I have not C. And I have not A.
And now, I can begin to just pick two clauses that I can resolve and apply
the resolution rule to them.
And so looking at these four clauses I see,
all right, these two clauses are ones I can resolve.
I can resolve them because there are complementary literals that
show up in them.
There's a C here, and a not C here.
So just looking at these two clauses, if I know that not B or C is true,
and I know that C is not true, well, then
I can resolve these two clauses to say, all right, not B. That must be true.
I can generate this new clause as a new piece of information
that I now know to be true.
And all right, now I can repeat this process.
Do the process again.
Can I use resolution again to get some new conclusion?
Well, it turns out I can.
I can use that new clause I just generated, along with this one here.
There are complementary literals.
This B is complementary to, or conflicts with this not B over here.
And so if I know that A or B is true, and I know that B is not true,
well, then the only remaining possibility is that A must be true.
So now we have A. That is a new clause that I've been able to generate.
And now I can do this one more time, and looking
for two clauses that can be resolved.
And you might programmatically do this by just looping
over all possible pairs of clauses and checking
for complementary literals in each.
And here I can say, all right, I found two clauses.
Not A and A that conflict with each other.
And when I resolve these two together, well, this
is the same as when we were resolving P and not P from before.
When I resolve these two clauses together, I get rid of the As,
and I'm left with the empty clause.
And the empty clause we know to be false, which means we
have a contradiction.
Which means we can safely say, that this whole knowledge
base does entail A. That if this sentence is true,
that we know that A, for sure, is also true.
So this now, using inference by resolution,
is an entirely different way to take some statement
and try and prove that it is, in fact, true.
Instead of enumerating all of the possible worlds
that we might be in in order to try to figure out in which case
is the knowledge base true, and in which case is our query true.
Instead we use this resolution algorithm to say,
let's keep trying to figure out what conclusions we can draw
and see if we reach a contradiction.
And if we reach a contradiction, then that
tells us something about whether our knowledge actually entails the query
or not.
And it turns out there are many different algorithms that
can be used for inference.
What we've just looked at here are just a couple of them.
And, in fact, all of this is just based on one particular type of logic.
It's based on propositional logic, where we have these individual symbols
and we connect them using and, and or, and not, and implies,
and biconditionals.
But propositional logic is not the only kind of logic that exists.
And in fact, we see that there are limitations
that exist in propositional logic, especially
as we saw in examples like with the Mastermind example,
or with the example with the logic puzzle
where we had different Hogwart's has people that belong to different houses,
and we were trying to figure out who belonged to which houses.
There were a lot of different propositional symbols
that we needed in order to represent some fairly basic ideas.
So now as a final topic that we'll take a look at,
just before we end class today, is one final type of logic,
different from propositional logic, known
as first order logic, which is a little bit more
powerful than propositional logic, and is
going to make it easier for us to express certain types of ideas.
In propositional logic, if we think back to that puzzle
with the people and the Hogwart's houses,
we had a whole bunch of symbols, and every symbol
could only be true or false.
We had a symbol for Minerva Gryffindor, which was either true
if Minerva was in Gryffindor, and false otherwise.
And likewise, for Minerva Hufflepuff, and Minerva Ravenclaw,
and Minerva Slytherin, and so forth.
But this was starting to get quite redundant.
That we wanted some way to be able to express
that there's a relationship between these propositional symbols.
That Minerva shows up in all of them.
And also, I would have liked to have not have
had so many different symbols to represent
what really was a fairly straightforward problem.
So first order logic will give us a different way
of trying to deal with this idea by giving us
two different types of symbols.
We're going to have constant symbols that are going to represent objects,
like people or houses.
And then predicate symbols, which you can
think of as relations or functions, that take an input and evaluate them to,
like, true or false, for example.
That tell us whether or not some property of some constant,
or some pair of constants, or multiple constants, actually holds.
So we'll see an example of that in just a moment.
But for now, in this same problem, our constant symbols might be objects.
Things like people or houses.
So Minerva, Pomona, Horace, Gildaroy.
Those are all constant symbols.
As are my four houses--
Gryffindor, Hufflepuff, Ravenclaw, and Slytherin.
Predicates, meanwhile, these predicate symbols
are going to be properties that might hold true or false
of these individual constants.
So person might hold true of Minerva, but it would be false for Gryffindor,
because Gryffindor is not a person.
And house is going to hold true for Ravenclaw,
but it's not going to hold true for Horace, for example,
because Horace is a person.
And belongs, meanwhile, is going to be some relation that is
going to relate people to their houses.
And it's going to only tell me when someone belongs to a house or does not.
So let's take a look at some examples of what a sentence in first order logic
might actually look like.
A sentence might look like something like this.
Person Minerva, with Minerva in parentheses.
And person being a predicate symbol, Minerva being a constant symbol.
This sentence in first order logic effectively means Minerva is a person,
or the person property applies to the Minerva object.
So if I want to say something like Minerva is a person,
here is how I express that idea using first order logic.
Meanwhile, I can say something like house Gryffindor
to, likewise, express the idea that Gryffindor is a house.
I can do that this way.
And all of the same logical connectives that we saw in propositional logic,
those are going to work here, too.
So and, or, implication, biconditional, not.
In fact, I can use not to say something like, not house Minerva.
And this sentence in first order logic means something like Minerva
is not a house.
It is not true that the house property applies to Minerva.
Meanwhile, in addition to some of these predicate symbols that
just take a single argument, some of our predicate symbols
are going to express binary relations.
Relations between two of its arguments.
So I could say something like, belongs to, and then two inputs,
Minerva and Gryffindor to express the idea
that Minerva belongs to Gryffindor.
And so now here's the key difference, or one of the key differences,
between this and propositional logic.
In propositional logic, I needed one symbol for Minerva Gryffindor,
and one symbol for Minerva Hufflepuff, and one symbol
for all the other people's Gryffindor and Hufflepuff variables.
In this case, I just need one symbol for each of my people,
and one symbol for each of my houses, and then I
can express, as a predicate, something like, belongs to,
and say, belongs to Minerva Gryffindor to express the idea
that Minerva belongs to Gryffindor house.
So already we can see that first order logic
is quite expressive in being able to express these sorts of sentences using
the existing constant symbols and predicates that
already exist, while minimizing the number of new symbols
that I need to create.
I can just use eight symbols for people, for houses, instead
of 16 symbols for every possible combination of each.
But first order logic gives us a couple of additional features
that we can use to express even more complex ideas.
And these additional features are generally known as quantifiers.
And there are two main quantifiers in first order logic.
The first of which is universal quantification.
Universal quantification lets me express an idea,
like something is going to be true for all values of a variable.
Like for all values of x, some statement is going to hold true.
So what might a sentence in universal quantification look like?
Well, we're going to use this upside-down A to mean for all.
So upside-down Ax means for all values of x, where x is any object,
this is going to hold true.
Belongs to x Gryffindor implies not belongs to x Hufflepuff.
So let's try and parse this out.
This means that for all values of x, If this holds true,
if x belongs to Gryffindor, then this does not hold true.
X does not belong to Hufflepuff.
So translated into English, this sentence
is saying something like, for all objects x, if x belongs to Gryffindor,
then x does not belong to Hufflepuff, for example.
Or phrased even more simply, anyone in Gryffindor is not a Hufflepuff.
A simplified way of saying the same thing.
So this universal quantification lets us express an idea,
like something is going to hold true for all values of a particular variable.
In addition to universal quantification, though, we also
have existential quantification.
Whereas universal quantification said that something
is going to be true for all values of variable,
existential quantification says that some expression
is going to be true for some value of a variable.
At least one value of the variable.
So let's take a look at a sample sentence using
existential quantification.
One such sentence looks like this.
There exists an x-- this backwards E stands for exists.
And here we're saying there exists an x, such that house x
and belongs to Minerva x.
In other words, there exists some object x, where x is a house,
and Minerva belongs to x.
Or phrased a little more succinctly in English, you're just saying,
Minerva belongs to a house.
There's some object that is a house, and Minerva belongs to a house.
And combining this universal and existential quantification,
we can create far more sophisticated logical statements
than we were able to just using propositional logic.
I could combine these to say something like this.
For all x, person x implies there exists a y, such that house y
and belongs to xy.
So a lot of stuff going on there.
A lot of symbols.
Let's try and parse it out and just understand what it's saying.
Here we're saying that for all values of x, if x is a person, then this is true.
So in other words, I'm saying for all people,
and we call that person x, this statement is going to be true.
What statement is true of all people?
Well, there exists a y that is the house, so there exists some house,
and x belongs to y.
In other words I'm saying, that for all people out there,
there exists some house, such that x, the person, belongs to y, the house.
So say it more succinctly, I'm saying that every person belongs to a house.
That for all x, if x is a person, then there exists a house that x belongs to.
And so we can now express a lot more powerful ideas using this idea now
of first order logic.
And it turns out there are many other kinds of logic out there.
There's second order logic and other higher order logic, each of which
allows us to express more and more complex ideas.
But all of it, in this case, is really in pursuit of the same goal, which
is the representation of knowledge.
We want our AI agents to be able to know information,
to represent that information, whether that's
using propositional logic, or first order logic, or some other logic.
And then be able to reason based on that.
To be able to draw conclusions.
Make inferences.
Figure out whether there's some sort of entailment relationship,
as by using some sort of inference algorithm.
Something like inference by resolution, or model checking,
or any number of these other algorithms that we
can use in order to take information that we know
and translate it to additional conclusions.
So all of this has helped us to create AI
that is able to represent information about what it knows
and what it doesn't know.
Next time though, we'll take a look at how
we can make our AI even more powerful by not just encoding information
that we know for sure to be true, and not to be true, but also,
to take a look at uncertainty.
To look at what happens if AI thinks that something might be probable,
or maybe not very probable, or somewhere in between those two extremes,
all in the pursuit of trying to build our intelligent systems to be even more
intelligent.
We'll see you next time.