1 00:00:00,000 --> 00:00:03,912 [MUSIC PLAYING] 2 00:00:03,912 --> 00:00:17,707 3 00:00:17,707 --> 00:00:18,540 BRIAN YU: All right. 4 00:00:18,540 --> 00:00:21,420 Welcome back, everyone, to an introduction to artificial intelligence 5 00:00:21,420 --> 00:00:22,470 with Python. 6 00:00:22,470 --> 00:00:24,510 Last time we took a look at search problems 7 00:00:24,510 --> 00:00:26,820 in particular, where we have AI agents that 8 00:00:26,820 --> 00:00:29,760 are trying to solve some sort of problem by taking actions 9 00:00:29,760 --> 00:00:32,009 in some sort of environment, whether that environment 10 00:00:32,009 --> 00:00:34,730 is trying to take actions by playing moves in a game, 11 00:00:34,730 --> 00:00:37,980 or whether those actions are something like trying to figure out where to make 12 00:00:37,980 --> 00:00:41,983 turns in order to get driving directions from point A to point B. 13 00:00:41,983 --> 00:00:44,400 This time we're going to turn our attention more generally 14 00:00:44,400 --> 00:00:46,470 to just this idea of knowledge. 15 00:00:46,470 --> 00:00:49,620 The idea that a lot of intelligence is based on knowledge, especially 16 00:00:49,620 --> 00:00:51,510 if we think about human intelligence. 17 00:00:51,510 --> 00:00:53,130 People know information. 18 00:00:53,130 --> 00:00:54,900 We know facts about the world. 19 00:00:54,900 --> 00:00:57,030 And using that information that we know, we're 20 00:00:57,030 --> 00:00:59,820 able to draw conclusions-- reason about the information 21 00:00:59,820 --> 00:01:03,570 that we know in order to figure out how to do something or figure out 22 00:01:03,570 --> 00:01:07,230 some other piece of information that we conclude based on the information we 23 00:01:07,230 --> 00:01:09,360 already have available to us. 24 00:01:09,360 --> 00:01:13,650 What we'd like to focus on now is the ability to take this idea of knowledge 25 00:01:13,650 --> 00:01:15,630 and being able to reason based on knowledge, 26 00:01:15,630 --> 00:01:18,570 and apply those ideas to artificial intelligence. 27 00:01:18,570 --> 00:01:20,520 In particular, we're going to be building what 28 00:01:20,520 --> 00:01:22,980 are known as knowledge-based agents. 29 00:01:22,980 --> 00:01:27,340 Agents that are able to reason and act by representing knowledge internally. 30 00:01:27,340 --> 00:01:30,240 Somehow inside of our AI, they have some understanding 31 00:01:30,240 --> 00:01:32,250 of what it means to know something. 32 00:01:32,250 --> 00:01:34,890 And ideally, they have some algorithms, or some techniques 33 00:01:34,890 --> 00:01:37,740 they can use based on that knowledge that they know in order 34 00:01:37,740 --> 00:01:40,980 to figure out the solution to a problem, or figure out 35 00:01:40,980 --> 00:01:45,000 some additional piece of information that can be helpful in some sense. 36 00:01:45,000 --> 00:01:47,400 So what do we mean by reasoning based on knowledge 37 00:01:47,400 --> 00:01:48,960 to be able to draw conclusions? 38 00:01:48,960 --> 00:01:52,260 Well, let's look at a simple example drawn from the world of Harry Potter. 39 00:01:52,260 --> 00:01:55,890 We take one sentence that we know to be true. 40 00:01:55,890 --> 00:01:59,370 If it didn't rain, then Harry visited Hagrid today. 41 00:01:59,370 --> 00:02:02,150 So one fact that we might know about the world. 42 00:02:02,150 --> 00:02:03,480 And then we take another fact. 43 00:02:03,480 --> 00:02:07,081 Harry visited Hagrid or Dumbledore today, but not both. 44 00:02:07,081 --> 00:02:08,789 So it tells us something about the world. 45 00:02:08,789 --> 00:02:11,490 That Harry either visited Hagrid but not Dumbledore, 46 00:02:11,490 --> 00:02:13,910 or Harry visited Dumbledore but not Hagrid. 47 00:02:13,910 --> 00:02:16,410 And now we have a third piece of information about the world 48 00:02:16,410 --> 00:02:19,020 that Harry visited Dumbledore today. 49 00:02:19,020 --> 00:02:21,240 So we now have three pieces of information now. 50 00:02:21,240 --> 00:02:25,872 Three facts inside of a knowledge base, so to speak-- information that we know. 51 00:02:25,872 --> 00:02:28,080 And now we, as humans, can try and reason about this, 52 00:02:28,080 --> 00:02:30,720 and figure out based on this information, what 53 00:02:30,720 --> 00:02:33,570 additional information can we begin to conclude? 54 00:02:33,570 --> 00:02:35,730 And well, looking at these last two statements, 55 00:02:35,730 --> 00:02:39,570 Harry either visited Hagrid or Dumbledore but not both, 56 00:02:39,570 --> 00:02:41,882 and we know that Harry visited Dumbledore today. 57 00:02:41,882 --> 00:02:43,590 Well, then it's pretty reasonable that we 58 00:02:43,590 --> 00:02:46,620 could draw the conclusion that, you know what, Harry must not 59 00:02:46,620 --> 00:02:48,120 have visited Hagrid today. 60 00:02:48,120 --> 00:02:50,790 Because based on a combination of these two statements, 61 00:02:50,790 --> 00:02:53,010 we can draw this inference, so to speak. 62 00:02:53,010 --> 00:02:56,282 A conclusion that Harry did not visit Hagrid today. 63 00:02:56,282 --> 00:02:59,490 But it turns out we can even do a little bit better than that-- get some more 64 00:02:59,490 --> 00:03:02,010 information-- by taking a look at this first statement 65 00:03:02,010 --> 00:03:03,480 and reasoning about that. 66 00:03:03,480 --> 00:03:08,222 This first statement says, if it didn't rain, then Harry visited Hagrid today. 67 00:03:08,222 --> 00:03:09,180 So what does that mean? 68 00:03:09,180 --> 00:03:13,350 In all cases where it didn't rain, then we know that Harry visited Hagrid. 69 00:03:13,350 --> 00:03:16,950 But if we also know now that Harry did not visit Hagrid, 70 00:03:16,950 --> 00:03:19,743 then it tells us something about our initial premise 71 00:03:19,743 --> 00:03:20,910 that we were thinking about. 72 00:03:20,910 --> 00:03:24,330 In particular, it tells us that it did rain today. 73 00:03:24,330 --> 00:03:28,560 Because we can reason if it didn't rain, that Harry would have visited Hagrid. 74 00:03:28,560 --> 00:03:33,150 But we know for a fact that Harry did not visit Hagrid today. 75 00:03:33,150 --> 00:03:36,060 So it's this kind of reasoning, the sort of logical reasoning 76 00:03:36,060 --> 00:03:38,250 where we use logic based on the information 77 00:03:38,250 --> 00:03:42,113 that we know in order to take information and reach conclusions. 78 00:03:42,113 --> 00:03:45,280 That is going to be the focus of what we're going to be talking about today. 79 00:03:45,280 --> 00:03:48,810 How can we make our artificial intelligence logical so 80 00:03:48,810 --> 00:03:52,860 that they can perform the same kinds of deduction, the same kinds of reasoning 81 00:03:52,860 --> 00:03:54,960 that we've been doing so far. 82 00:03:54,960 --> 00:03:59,070 Of course, humans reason about logic generally in terms of human language. 83 00:03:59,070 --> 00:04:02,490 That I just now was speaking in English, talking in English, 84 00:04:02,490 --> 00:04:05,280 about these sentences and trying to reason through how 85 00:04:05,280 --> 00:04:06,900 it is that they relate to one another. 86 00:04:06,900 --> 00:04:09,300 We're going to need to be a little bit more formal when 87 00:04:09,300 --> 00:04:11,760 we turn our attention to computers and being 88 00:04:11,760 --> 00:04:15,900 able to encode this notion of logic, and truthhood and falsehood inside 89 00:04:15,900 --> 00:04:17,050 of a machine. 90 00:04:17,050 --> 00:04:20,310 So we're going to need to introduce a few more terms and a few symbols that 91 00:04:20,310 --> 00:04:22,740 will help us reason through this idea of logic 92 00:04:22,740 --> 00:04:24,780 inside of an artificial intelligence. 93 00:04:24,780 --> 00:04:27,107 And we'll begin with the idea of a sentence. 94 00:04:27,107 --> 00:04:29,190 Now, a sentence in a natural language like English 95 00:04:29,190 --> 00:04:32,340 is just something that I'm saying, like what I'm saying right now. 96 00:04:32,340 --> 00:04:37,190 In the context of AI though, a sentence is just an assertion about the world 97 00:04:37,190 --> 00:04:41,040 in what we're going to call a knowledge representation language, 98 00:04:41,040 --> 00:04:45,413 some way of representing knowledge inside of our computers. 99 00:04:45,413 --> 00:04:47,580 And the way that we're going to spend most of today, 100 00:04:47,580 --> 00:04:49,710 reasoning about knowledge, is through a type 101 00:04:49,710 --> 00:04:51,932 of logic known as propositional logic. 102 00:04:51,932 --> 00:04:55,140 There are a number of different types of logic, some of which we'll touch on. 103 00:04:55,140 --> 00:04:59,280 But propositional logic is based on a logic of propositions, or just 104 00:04:59,280 --> 00:05:01,090 statements about the world. 105 00:05:01,090 --> 00:05:03,510 And so we begin in propositional logic with the notion 106 00:05:03,510 --> 00:05:05,400 of propositional symbols. 107 00:05:05,400 --> 00:05:08,430 We will have certain symbols that are oftentimes just letters, 108 00:05:08,430 --> 00:05:12,090 something like P or Q or R, where each of those symbols 109 00:05:12,090 --> 00:05:16,200 is going to represent some fact or sentence about the world. 110 00:05:16,200 --> 00:05:20,160 So P, for example, might represent the fact that it is raining. 111 00:05:20,160 --> 00:05:23,550 And so P is going to be a symbol that represents that idea. 112 00:05:23,550 --> 00:05:27,300 And Q, for example, might represent Harry visited Hagrid today. 113 00:05:27,300 --> 00:05:30,930 Each of these propositional symbols represents some sentence 114 00:05:30,930 --> 00:05:33,660 or some fact about the world. 115 00:05:33,660 --> 00:05:36,750 But in addition to just having individual facts about the world, 116 00:05:36,750 --> 00:05:40,380 we want some way to connect these propositional symbols together 117 00:05:40,380 --> 00:05:44,880 in order to reason more complexly about other facts that might exist inside 118 00:05:44,880 --> 00:05:46,560 of the world in which we're reasoning. 119 00:05:46,560 --> 00:05:49,740 So in order to do that, we'll need to introduce some additional symbols that 120 00:05:49,740 --> 00:05:51,930 are known as logical connectives. 121 00:05:51,930 --> 00:05:54,150 Now, there are a number of these logical connectives, 122 00:05:54,150 --> 00:05:57,275 but five of the most important, and the ones we're going to focus on today, 123 00:05:57,275 --> 00:06:00,840 are these five up here, each represented by a logical symbol. 124 00:06:00,840 --> 00:06:03,690 Not is represented by this symbol here. 125 00:06:03,690 --> 00:06:07,050 And is represented as sort of an upside-down V. Or 126 00:06:07,050 --> 00:06:08,970 is represented by a V shape. 127 00:06:08,970 --> 00:06:11,910 Implication-- and we'll talk about what that means in just a moment-- 128 00:06:11,910 --> 00:06:13,680 is represented by an arrow. 129 00:06:13,680 --> 00:06:16,850 And biconditional-- again, we'll talk about what that means in a moment-- 130 00:06:16,850 --> 00:06:18,900 is represented by these double arrows. 131 00:06:18,900 --> 00:06:21,570 But these five logical connectives are the main ones 132 00:06:21,570 --> 00:06:24,630 we're going to be focusing on in terms of thinking about how 133 00:06:24,630 --> 00:06:27,270 it is that a computer can reason about facts 134 00:06:27,270 --> 00:06:30,760 and draw conclusions based on the facts that it knows. 135 00:06:30,760 --> 00:06:32,760 But in order to get there we need to take a look 136 00:06:32,760 --> 00:06:36,480 at each of these logical connectives and build up an understanding for what 137 00:06:36,480 --> 00:06:38,400 it is that they actually mean. 138 00:06:38,400 --> 00:06:40,950 So let's go ahead and begin with the not symbol. 139 00:06:40,950 --> 00:06:42,570 So this not symbol here. 140 00:06:42,570 --> 00:06:45,510 And what we're going to show for each of these logical connectives 141 00:06:45,510 --> 00:06:47,670 is what we're going to call a truth table. 142 00:06:47,670 --> 00:06:50,280 A table that demonstrates what this word not 143 00:06:50,280 --> 00:06:55,080 means when we attach it to a propositional symbol or any sentence 144 00:06:55,080 --> 00:06:56,910 inside of our logical language. 145 00:06:56,910 --> 00:07:01,200 And so the truth table for not is shown right here. 146 00:07:01,200 --> 00:07:04,680 If P-- some propositional symbol or some other sentence, even-- 147 00:07:04,680 --> 00:07:08,940 is false, then not P is true. 148 00:07:08,940 --> 00:07:13,290 And if P is true, then not P is false. 149 00:07:13,290 --> 00:07:15,540 So you can imagine placing this not symbol 150 00:07:15,540 --> 00:07:18,420 in front of some sentence of propositional logic. 151 00:07:18,420 --> 00:07:20,460 Just says the opposite of that. 152 00:07:20,460 --> 00:07:24,280 So if, for example, P represented it is raining, 153 00:07:24,280 --> 00:07:28,230 then not P would represent the idea that it is not raining. 154 00:07:28,230 --> 00:07:32,820 And as you might expect, if P is false, meaning if the sentence it is raining 155 00:07:32,820 --> 00:07:37,200 is false, well, then the sentence not P must be true. 156 00:07:37,200 --> 00:07:40,530 The sentence that it is not raining is, therefore, true. 157 00:07:40,530 --> 00:07:44,310 So not, you can imagine, just takes whatever is in P and it inverts it. 158 00:07:44,310 --> 00:07:47,730 It turns false into true, and true into false. 159 00:07:47,730 --> 00:07:50,820 Much analogously to what the English word not means. 160 00:07:50,820 --> 00:07:55,500 Just taking whatever comes after it and inverting it to mean the opposite. 161 00:07:55,500 --> 00:07:58,080 Next up, and also very English-like, is this idea 162 00:07:58,080 --> 00:08:02,460 of and, represented by this upside-down V shape, or this point shape. 163 00:08:02,460 --> 00:08:05,760 And as opposed to just taking a single argument the way not does-- 164 00:08:05,760 --> 00:08:08,100 we have P and we have not P-- 165 00:08:08,100 --> 00:08:11,370 and is going to combine two different sentences 166 00:08:11,370 --> 00:08:13,440 and propositional logic together. 167 00:08:13,440 --> 00:08:16,760 So I might have one sentence P and another sentence Q, 168 00:08:16,760 --> 00:08:21,110 and I want to combine them together to say P and Q. 169 00:08:21,110 --> 00:08:24,300 And the general logic for what P and Q means is it 170 00:08:24,300 --> 00:08:26,790 means that both of its operands are true. 171 00:08:26,790 --> 00:08:30,910 P is true, and also, Q is true. 172 00:08:30,910 --> 00:08:33,480 And so here's what that truth table looks like. 173 00:08:33,480 --> 00:08:36,360 This time we have two variables, P and Q. 174 00:08:36,360 --> 00:08:39,840 And when we have two variables, each of which can be in two possible states, 175 00:08:39,840 --> 00:08:42,820 true or false, that leads to two squared, 176 00:08:42,820 --> 00:08:46,890 or four, possible combinations of truth and falsehood. 177 00:08:46,890 --> 00:08:49,320 So we have P is false and Q is false. 178 00:08:49,320 --> 00:08:51,460 We have P is false and Q is true. 179 00:08:51,460 --> 00:08:53,040 P is true and Q is false. 180 00:08:53,040 --> 00:08:55,510 And then P and Q both are true. 181 00:08:55,510 --> 00:08:57,300 And those are the only four possibilities 182 00:08:57,300 --> 00:08:59,650 for what P and Q could mean. 183 00:08:59,650 --> 00:09:03,750 And in each of those situations, this third column here, P and Q, 184 00:09:03,750 --> 00:09:08,100 is telling us a little bit about what it actually means for P and Q to be true. 185 00:09:08,100 --> 00:09:12,360 And we see that the only case where P and Q is true is in this fourth row 186 00:09:12,360 --> 00:09:17,160 here, where P happens to be true, Q also happens to be true. 187 00:09:17,160 --> 00:09:22,320 And in all other situations, P and Q is going to evaluate to false. 188 00:09:22,320 --> 00:09:25,920 So this, again, is much in line with what our intuition of and might mean. 189 00:09:25,920 --> 00:09:33,620 If I say P and Q, I probably mean that I expect both P and Q to be true. 190 00:09:33,620 --> 00:09:36,620 Next up, also potentially consistent with what we mean, 191 00:09:36,620 --> 00:09:39,680 is this word or, represented by this V shape, sort 192 00:09:39,680 --> 00:09:42,020 of an upside-down and symbol. 193 00:09:42,020 --> 00:09:45,890 And or, as the name might suggest, is true if either of its arguments 194 00:09:45,890 --> 00:09:46,550 are true. 195 00:09:46,550 --> 00:09:51,780 As long as P is true or Q is true, then P or Q is going to be true. 196 00:09:51,780 --> 00:09:56,960 Which means the only time that P or Q is false is if both of its operands 197 00:09:56,960 --> 00:09:57,710 are false. 198 00:09:57,710 --> 00:10:03,050 If P is false and Q is false, then P or Q is going to be false. 199 00:10:03,050 --> 00:10:07,430 But in all other cases, if at least one of the operands is true-- 200 00:10:07,430 --> 00:10:08,880 maybe they're both true-- 201 00:10:08,880 --> 00:10:12,890 in which case P or Q is going to evaluate to true. 202 00:10:12,890 --> 00:10:16,760 Now, this is mostly consistent with the way that most people might use the word 203 00:10:16,760 --> 00:10:19,603 or, in the sense of speaking the word or in normal English. 204 00:10:19,603 --> 00:10:21,770 Though there is sometimes when we might say or where 205 00:10:21,770 --> 00:10:24,620 we mean P or Q, but not both. 206 00:10:24,620 --> 00:10:27,770 Or we mean sort of it can only be one or the other. 207 00:10:27,770 --> 00:10:33,220 It's important to note that this symbol here, this or, means P or Q or both. 208 00:10:33,220 --> 00:10:34,640 That those are totally OK. 209 00:10:34,640 --> 00:10:38,210 As long as either or both of them are true, then the or 210 00:10:38,210 --> 00:10:40,680 is going to evaluate to be true as well. 211 00:10:40,680 --> 00:10:43,040 It's only in the case where all of the operands 212 00:10:43,040 --> 00:10:47,570 are false that P or Q ultimately evaluates to false as well. 213 00:10:47,570 --> 00:10:51,200 In logic there's another symbol known as the exclusive or, which 214 00:10:51,200 --> 00:10:55,188 encodes this idea of exclusivity of, like, one or the other, but not both. 215 00:10:55,188 --> 00:10:57,230 But we're not going to be focusing on that today. 216 00:10:57,230 --> 00:11:01,110 Whenever we talk about or, we're always talking about either or both, 217 00:11:01,110 --> 00:11:05,830 in this case, as represented by this truth table here. 218 00:11:05,830 --> 00:11:09,140 So that now is not, and an, and or. 219 00:11:09,140 --> 00:11:12,120 And next step is what we might call implication, as denoted 220 00:11:12,120 --> 00:11:13,560 by this arrow symbol. 221 00:11:13,560 --> 00:11:16,860 So we have P and Q. And this sentence here 222 00:11:16,860 --> 00:11:20,820 will generally read as P implies Q. And what 223 00:11:20,820 --> 00:11:27,790 P implies Q means is that if P is true, then Q is also true. 224 00:11:27,790 --> 00:11:32,040 So I might say something like, if it is raining, then I will be indoors. 225 00:11:32,040 --> 00:11:36,140 Meaning it is raining implies I will be indoors 226 00:11:36,140 --> 00:11:38,760 is the logical sentence that I'm saying there. 227 00:11:38,760 --> 00:11:42,010 And the truth table for this can sometimes be a little bit tricky. 228 00:11:42,010 --> 00:11:48,570 So obviously, if P is true and Q is true, then P implies Q, that's true. 229 00:11:48,570 --> 00:11:50,400 That definitely makes sense. 230 00:11:50,400 --> 00:11:55,030 And it should also stand to reason that when P is true and Q is false, 231 00:11:55,030 --> 00:11:56,880 then P implies Q is false. 232 00:11:56,880 --> 00:12:01,680 Because if I said to you, if it is raining, then I will be indoors, 233 00:12:01,680 --> 00:12:05,640 and it is raining, but I'm not indoors, well, then it 234 00:12:05,640 --> 00:12:08,940 would seem to be that my original statement was not true. 235 00:12:08,940 --> 00:12:13,620 P implies Q means that if P is true, then Q also needs to be true. 236 00:12:13,620 --> 00:12:17,670 And if it's not, well, then the statement is false. 237 00:12:17,670 --> 00:12:21,840 Also worth noting, though, is what happens when P is false. 238 00:12:21,840 --> 00:12:26,580 When P is false, the implication makes no claim at all. 239 00:12:26,580 --> 00:12:30,960 If I say something like, if it is raining, then I will be indoors, 240 00:12:30,960 --> 00:12:33,720 and it turns out it's not raining, then in that case, 241 00:12:33,720 --> 00:12:35,910 I am not making any statement as to whether 242 00:12:35,910 --> 00:12:38,130 or not I will be indoors or not. 243 00:12:38,130 --> 00:12:41,880 P implies Q just means that if P is true, Q must be true. 244 00:12:41,880 --> 00:12:44,940 But if P is not true, then we make no claim about 245 00:12:44,940 --> 00:12:47,440 whether or not Q is true at all. 246 00:12:47,440 --> 00:12:50,610 So in either case, if P is false, it doesn't matter 247 00:12:50,610 --> 00:12:52,440 what Q is, whether it's false or true. 248 00:12:52,440 --> 00:12:54,900 We're not making any claim about Q whatsoever. 249 00:12:54,900 --> 00:12:58,060 We can still evaluate the implication to true. 250 00:12:58,060 --> 00:13:00,920 The only way that the implication is ever false 251 00:13:00,920 --> 00:13:05,250 is if our premise, P is true, but the conclusion that we're drawing, 252 00:13:05,250 --> 00:13:07,360 Q, happens to be false. 253 00:13:07,360 --> 00:13:13,710 So in that case, we would say P does not imply Q in that case. 254 00:13:13,710 --> 00:13:17,520 Finally, the last connective that we'll discuss is this biconditional. 255 00:13:17,520 --> 00:13:21,910 You can think of a biconditional as a condition that goes in both directions. 256 00:13:21,910 --> 00:13:24,850 So originally, when I said something like, if it is raining, 257 00:13:24,850 --> 00:13:26,880 then I will be indoors. 258 00:13:26,880 --> 00:13:29,280 I didn't say what would happen if it wasn't raining. 259 00:13:29,280 --> 00:13:31,680 Maybe I'll be indoors, maybe I'll be outdoors. 260 00:13:31,680 --> 00:13:35,760 This biconditional you can read as an if and only if. 261 00:13:35,760 --> 00:13:41,280 So I can say, I will be indoors if and only if it is raining. 262 00:13:41,280 --> 00:13:43,920 Meaning if it is raining, then I will be indoors. 263 00:13:43,920 --> 00:13:47,910 And if I am indoors, it's reasonable to conclude that it is also raining. 264 00:13:47,910 --> 00:13:52,960 So this biconditional is only true when P and Q are the same. 265 00:13:52,960 --> 00:13:57,750 So if P is true and Q is true, then this biconditional is also true-- 266 00:13:57,750 --> 00:14:00,360 P implies Q. But also the reverse is true. 267 00:14:00,360 --> 00:14:05,572 Q also implies P. So if P and Q both happen to be false, 268 00:14:05,572 --> 00:14:06,780 we would still say it's true. 269 00:14:06,780 --> 00:14:10,050 But in any of these other two situations, this P if and only 270 00:14:10,050 --> 00:14:13,260 if Q is going to ultimately evaluate to false. 271 00:14:13,260 --> 00:14:15,540 So a lot of trues and falses going on there, 272 00:14:15,540 --> 00:14:18,270 but these five basic logical connectives are 273 00:14:18,270 --> 00:14:21,300 going to form the core of the language of propositional logic, 274 00:14:21,300 --> 00:14:24,780 the language that we're going to use in order to describe ideas, 275 00:14:24,780 --> 00:14:28,200 and the language that we're going to use in order to reason about those ideas 276 00:14:28,200 --> 00:14:30,970 in order to draw conclusions. 277 00:14:30,970 --> 00:14:33,330 So let's now take a look at some of the additional terms 278 00:14:33,330 --> 00:14:35,163 that we'll need to know about in order to go 279 00:14:35,163 --> 00:14:38,070 about trying to form this language of propositional logic, 280 00:14:38,070 --> 00:14:41,940 and writing AI that's actually able to understand this sort of logic. 281 00:14:41,940 --> 00:14:43,950 The next thing we're going to need is the notion 282 00:14:43,950 --> 00:14:46,930 of what is actually true about the world. 283 00:14:46,930 --> 00:14:49,050 We have a whole bunch of propositional symbols-- 284 00:14:49,050 --> 00:14:51,210 P and Q and R and maybe others. 285 00:14:51,210 --> 00:14:54,450 But we need some way of knowing, like, what actually is true in the world. 286 00:14:54,450 --> 00:14:58,680 Is P true or false, is Q true or false, so on and so forth. 287 00:14:58,680 --> 00:15:01,870 And to do that, we'll introduce the notion of a model. 288 00:15:01,870 --> 00:15:07,590 A model just assigns a truth value where a truth value is either true or false 289 00:15:07,590 --> 00:15:10,120 to every propositional symbol. 290 00:15:10,120 --> 00:15:13,650 In other words, it's creating what we might call a possible world. 291 00:15:13,650 --> 00:15:15,250 So let me give an example. 292 00:15:15,250 --> 00:15:18,090 If, for example, I have two propositional symbols, 293 00:15:18,090 --> 00:15:22,230 P is it is raining, and Q is it is a Tuesday, 294 00:15:22,230 --> 00:15:25,350 a model just takes each of these two symbols 295 00:15:25,350 --> 00:15:29,050 and assigns a truth value to them, either true or false. 296 00:15:29,050 --> 00:15:30,460 So here's a sample model. 297 00:15:30,460 --> 00:15:33,720 In this model, in other words, in this possible world, 298 00:15:33,720 --> 00:15:38,320 it is possible that P is true, meaning it is raining, and Q is false, 299 00:15:38,320 --> 00:15:40,260 meaning it is not a Tuesday. 300 00:15:40,260 --> 00:15:43,660 But there are other possible worlds or other models as well. 301 00:15:43,660 --> 00:15:46,230 There is some model where both of these variables are true. 302 00:15:46,230 --> 00:15:48,610 Some model where both of these variables are false. 303 00:15:48,610 --> 00:15:52,620 In fact, if there are N variables that are propositional symbols like this, 304 00:15:52,620 --> 00:15:56,010 that are either true or false, then the number of possible models 305 00:15:56,010 --> 00:15:59,880 is two to the N, because each of these possible models-- 306 00:15:59,880 --> 00:16:02,250 possible variables within my model could be 307 00:16:02,250 --> 00:16:08,130 set to either true or false, if I don't know any information about it. 308 00:16:08,130 --> 00:16:09,803 So now that I have the symbols-- 309 00:16:09,803 --> 00:16:11,720 the symbols and the connectives that I'm going 310 00:16:11,720 --> 00:16:15,380 to need in order to construct these parts of knowledge, 311 00:16:15,380 --> 00:16:17,690 we need some way to represent that knowledge. 312 00:16:17,690 --> 00:16:20,180 And to do so, we're going to allow our AI access 313 00:16:20,180 --> 00:16:22,700 to what we'll call a knowledge base. 314 00:16:22,700 --> 00:16:26,180 And a knowledge base is really just a set of sentences 315 00:16:26,180 --> 00:16:28,590 that our AI knows to be true. 316 00:16:28,590 --> 00:16:31,460 Some set of sentences in propositional logic 317 00:16:31,460 --> 00:16:35,340 that are things that our AI knows about the world. 318 00:16:35,340 --> 00:16:38,510 And so we might tell our AI some information, information 319 00:16:38,510 --> 00:16:40,910 about a situation that it finds itself in, 320 00:16:40,910 --> 00:16:44,135 or situation about a problem that it happens to be trying to solve. 321 00:16:44,135 --> 00:16:46,010 And we would give that information to the AI, 322 00:16:46,010 --> 00:16:49,190 that the AI would store inside of its knowledge base. 323 00:16:49,190 --> 00:16:51,710 And what happens next is the AI would like 324 00:16:51,710 --> 00:16:54,140 to use that information in the knowledge base 325 00:16:54,140 --> 00:16:57,810 to be able to draw conclusions about the rest of the world. 326 00:16:57,810 --> 00:16:59,780 And what do those conclusions look like? 327 00:16:59,780 --> 00:17:01,613 Well, to understand those conclusions, we'll 328 00:17:01,613 --> 00:17:04,400 need to introduce one more idea, one more symbol, and that 329 00:17:04,400 --> 00:17:06,890 is the notion of entailment. 330 00:17:06,890 --> 00:17:10,803 So this sentence here, with this double turnstile and these Greek letters-- 331 00:17:10,803 --> 00:17:13,220 this is the Greek letter alpha and the Greek letter beta-- 332 00:17:13,220 --> 00:17:17,180 and we read this as alpha entails beta. 333 00:17:17,180 --> 00:17:21,589 And alpha and beta here are just sentences in propositional logic. 334 00:17:21,589 --> 00:17:27,150 And what this means is that alpha entails beta means that in every model, 335 00:17:27,150 --> 00:17:31,640 in other words, in every possible world in which sentence a is true-- 336 00:17:31,640 --> 00:17:35,880 or sentence alpha is true, then sentence beta is also true. 337 00:17:35,880 --> 00:17:39,830 So if something entails something else, if alpha entails beta, 338 00:17:39,830 --> 00:17:42,500 it means that if I know alpha to be true, 339 00:17:42,500 --> 00:17:45,600 then beta must, therefore, also be true. 340 00:17:45,600 --> 00:17:52,160 So if my alpha is something like, I know that it is a Tuesday in January, 341 00:17:52,160 --> 00:17:54,440 then a reasonable beta might be something 342 00:17:54,440 --> 00:17:56,930 like, I know that it is January. 343 00:17:56,930 --> 00:17:59,810 Because in all worlds, where it is a Tuesday in January, 344 00:17:59,810 --> 00:18:03,500 I know for sure that it must be January, just by definition. 345 00:18:03,500 --> 00:18:06,230 This first statement, or sentence about the world, 346 00:18:06,230 --> 00:18:08,150 entails the second statement. 347 00:18:08,150 --> 00:18:11,720 And we can reasonably use deduction, based on that first sentence, 348 00:18:11,720 --> 00:18:16,640 to figure out that the second sentence is, in fact, true as well. 349 00:18:16,640 --> 00:18:19,130 And ultimately, it's this idea of entailment 350 00:18:19,130 --> 00:18:21,560 that we're going to try and encode into our computer. 351 00:18:21,560 --> 00:18:24,500 We want our AI agent to be able to figure out 352 00:18:24,500 --> 00:18:26,360 what the possible entailments are. 353 00:18:26,360 --> 00:18:30,380 We want our AI to be able to take these three sentences, sentences like, 354 00:18:30,380 --> 00:18:32,750 if it didn't rain, Harry visited Hagrid. 355 00:18:32,750 --> 00:18:35,750 That Harry visited Hagrid or Dumbledore but not both. 356 00:18:35,750 --> 00:18:37,460 And that Harry visited Dumbledore. 357 00:18:37,460 --> 00:18:40,340 And just using that information, we'd like our AI 358 00:18:40,340 --> 00:18:45,350 to be able to infer, or figure out, that using these three sentences inside 359 00:18:45,350 --> 00:18:48,380 of a knowledge base, we can draw some conclusions. 360 00:18:48,380 --> 00:18:50,780 In particular, we can draw the conclusions here 361 00:18:50,780 --> 00:18:53,810 that one, Harry did not visit Hagrid today. 362 00:18:53,810 --> 00:18:58,140 And we can draw the entailment two, that it did, in fact, rain today. 363 00:18:58,140 --> 00:19:00,130 And this process is known as inference. 364 00:19:00,130 --> 00:19:01,880 And that's what we're going to be focusing 365 00:19:01,880 --> 00:19:06,230 on today, this process of deriving new sentences from old ones. 366 00:19:06,230 --> 00:19:09,380 That I give you these three sentences, you put them in the knowledge base 367 00:19:09,380 --> 00:19:13,970 in, say, the AI, and the AI is able to use some sort of inference algorithm 368 00:19:13,970 --> 00:19:18,420 to figure out that these two sentences must also be true. 369 00:19:18,420 --> 00:19:21,000 And that is how we define inference. 370 00:19:21,000 --> 00:19:23,060 So let's take a look at an inference example 371 00:19:23,060 --> 00:19:26,840 to see how we might actually go about inferring things in a human sense, 372 00:19:26,840 --> 00:19:29,720 before we take a more algorithmic approach to see how we could 373 00:19:29,720 --> 00:19:31,670 encode this idea of inference in AI. 374 00:19:31,670 --> 00:19:35,240 And we'll see there are a number of ways that we can actually achieve this. 375 00:19:35,240 --> 00:19:38,210 So again, we'll deal with a couple of propositional symbols. 376 00:19:38,210 --> 00:19:41,930 We'll deal with P, Q and R. P is it is a Tuesday. 377 00:19:41,930 --> 00:19:43,490 Q is it is raining. 378 00:19:43,490 --> 00:19:45,410 And R is Harry will go for a run. 379 00:19:45,410 --> 00:19:48,920 Three propositional symbols that we are just defining to mean this. 380 00:19:48,920 --> 00:19:51,710 We're not saying anything yet about whether they're true or false. 381 00:19:51,710 --> 00:19:54,220 We're just defining what they are. 382 00:19:54,220 --> 00:19:58,280 Now we'll give ourselves, or an AI, access to a knowledge base, 383 00:19:58,280 --> 00:20:01,910 abbreviated to KB, to knowledge that we know about the world. 384 00:20:01,910 --> 00:20:03,710 We know this statement. 385 00:20:03,710 --> 00:20:05,210 All right, so let's try to parse it. 386 00:20:05,210 --> 00:20:07,252 The parentheses here are just used for precedent, 387 00:20:07,252 --> 00:20:09,530 so we can see what associates with what. 388 00:20:09,530 --> 00:20:15,763 But you would read this as P and not Q implies R. 389 00:20:15,763 --> 00:20:17,180 All right, so what does that mean? 390 00:20:17,180 --> 00:20:18,800 Let's put it piece by piece. 391 00:20:18,800 --> 00:20:21,140 P is it is a Tuesday. 392 00:20:21,140 --> 00:20:22,790 Q is it is raining. 393 00:20:22,790 --> 00:20:25,910 So not Q is it is not raining. 394 00:20:25,910 --> 00:20:29,360 And implies R is Harry will go for a run. 395 00:20:29,360 --> 00:20:33,140 So the way to read this entire sentence in human natural language at least, 396 00:20:33,140 --> 00:20:39,760 is if it is a Tuesday and it is not raining, then Harry will go for a run. 397 00:20:39,760 --> 00:20:43,820 So if it is a Tuesday and it is not raining, then Harry will go for a run. 398 00:20:43,820 --> 00:20:45,900 And that is now inside of our knowledge base. 399 00:20:45,900 --> 00:20:48,650 And let's now imagine that our knowledge base has two other pieces 400 00:20:48,650 --> 00:20:49,770 of information as well. 401 00:20:49,770 --> 00:20:54,020 It has information that P is true, that it is a Tuesday. 402 00:20:54,020 --> 00:20:58,160 And we also have the information not Q, that it is not raining. 403 00:20:58,160 --> 00:21:01,430 That this sentence Q, it is raining, happens to be false. 404 00:21:01,430 --> 00:21:04,130 And those are the three sentences that we have access to. 405 00:21:04,130 --> 00:21:08,680 P and not Q implies R, P and not Q. 406 00:21:08,680 --> 00:21:12,280 Using that information, we should be able to draw some inferences. 407 00:21:12,280 --> 00:21:18,410 P and not Q is only true if both P and not Q are true. 408 00:21:18,410 --> 00:21:20,510 Well, all right, we know that P is true. 409 00:21:20,510 --> 00:21:22,520 And we know that not Q is true. 410 00:21:22,520 --> 00:21:24,910 So we know that this whole expression is true. 411 00:21:24,910 --> 00:21:29,240 And the definition of implication is if this whole thing on the left is true, 412 00:21:29,240 --> 00:21:31,400 then this thing on the right must also be true. 413 00:21:31,400 --> 00:21:35,870 So if we know that P and not Q is true, then R must be true as well. 414 00:21:35,870 --> 00:21:38,470 So the inference we should be able to draw from all of this 415 00:21:38,470 --> 00:21:42,490 is that R is true, and we know that Harry will go for a run, 416 00:21:42,490 --> 00:21:46,330 by taking this knowledge inside of our knowledge base and being able to reason 417 00:21:46,330 --> 00:21:48,160 based on that idea. 418 00:21:48,160 --> 00:21:51,550 And so this ultimately, is the beginning of what we might consider 419 00:21:51,550 --> 00:21:53,800 to be some sort of inference algorithm. 420 00:21:53,800 --> 00:21:57,820 Some process that we can use to try and figure out whether or not 421 00:21:57,820 --> 00:21:59,380 we can draw some conclusion. 422 00:21:59,380 --> 00:22:02,380 And ultimately, what these inference algorithms are going to answer 423 00:22:02,380 --> 00:22:05,260 is the central question about entailment. 424 00:22:05,260 --> 00:22:07,090 Given some query about the world, something 425 00:22:07,090 --> 00:22:10,450 we're wondering about the world, and we'll call that query alpha, 426 00:22:10,450 --> 00:22:13,450 the question we want to ask, using these inference algorithms, 427 00:22:13,450 --> 00:22:19,100 is does it KB, our knowledge base, entail alpha? 428 00:22:19,100 --> 00:22:20,980 In other words, using only the information 429 00:22:20,980 --> 00:22:24,520 we know inside of our knowledge base, the knowledge that we have access to, 430 00:22:24,520 --> 00:22:28,540 can we conclude that this sentence alpha is true? 431 00:22:28,540 --> 00:22:31,190 And that's ultimately what we would like to do. 432 00:22:31,190 --> 00:22:32,480 So how can we do that? 433 00:22:32,480 --> 00:22:36,130 How can we go about writing an algorithm that can look at this knowledge base 434 00:22:36,130 --> 00:22:40,060 and figure out whether or not this query alpha is actually true? 435 00:22:40,060 --> 00:22:43,330 Well, it turns out there are a couple of different algorithms for doing so. 436 00:22:43,330 --> 00:22:47,470 And one of the simplest, perhaps, is known as model checking. 437 00:22:47,470 --> 00:22:49,960 Now remember, that a model is just some assignment 438 00:22:49,960 --> 00:22:53,470 of all of the propositional symbols inside of our language to a truth 439 00:22:53,470 --> 00:22:55,490 value, true or false. 440 00:22:55,490 --> 00:22:57,713 And you can think of a model as a possible world. 441 00:22:57,713 --> 00:23:00,130 That there are many possible worlds where different things 442 00:23:00,130 --> 00:23:01,430 might be true or false. 443 00:23:01,430 --> 00:23:03,260 And we can enumerate all of them. 444 00:23:03,260 --> 00:23:06,920 And the model checking algorithm does exactly that. 445 00:23:06,920 --> 00:23:08,950 So what does our model tracking algorithm do? 446 00:23:08,950 --> 00:23:11,530 Well, if we wanted to determine if our knowledge 447 00:23:11,530 --> 00:23:14,470 base entails some query alpha, then we are 448 00:23:14,470 --> 00:23:17,390 going to enumerate all possible models. 449 00:23:17,390 --> 00:23:21,030 In other words, consider all possible values of true and false 450 00:23:21,030 --> 00:23:21,820 for our variables. 451 00:23:21,820 --> 00:23:25,640 All possible states in which our world can be in. 452 00:23:25,640 --> 00:23:30,040 And if in every model where our knowledge base is true, 453 00:23:30,040 --> 00:23:34,777 alpha is also true, then we know that the knowledge base entails alpha. 454 00:23:34,777 --> 00:23:36,610 So let's take a closer look at that sentence 455 00:23:36,610 --> 00:23:38,527 and try and figure out what it actually means. 456 00:23:38,527 --> 00:23:42,430 If we know that in every model, in other words, in every possible world, 457 00:23:42,430 --> 00:23:45,820 no matter what assignment of true and false to variables you give, 458 00:23:45,820 --> 00:23:48,580 if we know that whenever our knowledge is true-- 459 00:23:48,580 --> 00:23:50,560 what we know to be true is true-- 460 00:23:50,560 --> 00:23:53,680 that this query alpha is also true. 461 00:23:53,680 --> 00:23:57,250 Well, then it stands to reason that as long as our knowledge base is true, 462 00:23:57,250 --> 00:24:00,152 then alpha must also be true. 463 00:24:00,152 --> 00:24:02,860 And so this is going to form the foundation of our model checking 464 00:24:02,860 --> 00:24:03,550 algorithm. 465 00:24:03,550 --> 00:24:06,010 We're going to enumerate all of the possible worlds, 466 00:24:06,010 --> 00:24:10,000 and ask ourselves, whenever the knowledge base is true, is alpha true? 467 00:24:10,000 --> 00:24:13,630 And if that's the case, then we know alpha to be true. 468 00:24:13,630 --> 00:24:15,820 And otherwise, there is no entailment. 469 00:24:15,820 --> 00:24:19,477 Our knowledge base does not entail alpha. 470 00:24:19,477 --> 00:24:21,310 All right, so this is a little bit abstract. 471 00:24:21,310 --> 00:24:25,260 But let's take a look at an example to try and put real propositional symbols 472 00:24:25,260 --> 00:24:26,560 to this idea. 473 00:24:26,560 --> 00:24:28,740 So again, we'll work with the same example. 474 00:24:28,740 --> 00:24:30,150 P is it is a Tuesday. 475 00:24:30,150 --> 00:24:31,470 Q is it is raining. 476 00:24:31,470 --> 00:24:33,510 R is Harry will go for a run. 477 00:24:33,510 --> 00:24:36,270 Our knowledge base contains these pieces of information. 478 00:24:36,270 --> 00:24:40,710 P and not Q implies R. We also know P, it is a Tuesday. 479 00:24:40,710 --> 00:24:43,140 And not Q, it is not raining. 480 00:24:43,140 --> 00:24:46,650 And our query, our alpha in this case, the thing we want to ask 481 00:24:46,650 --> 00:24:49,860 is R. We want to know, is it guaranteed? 482 00:24:49,860 --> 00:24:53,530 Is it entailed that Harry will go for a run. 483 00:24:53,530 --> 00:24:56,760 So the first step is to enumerate all of the possible models. 484 00:24:56,760 --> 00:24:59,400 We have three propositional symbols here, P, Q, 485 00:24:59,400 --> 00:25:04,110 and R, which means we have 2 to the third power, or 8 possible models. 486 00:25:04,110 --> 00:25:04,980 All false. 487 00:25:04,980 --> 00:25:06,010 False, false, true. 488 00:25:06,010 --> 00:25:07,080 False, true, false. 489 00:25:07,080 --> 00:25:08,310 False, true, true. 490 00:25:08,310 --> 00:25:08,970 Et cetera. 491 00:25:08,970 --> 00:25:13,800 Eight possible ways you could assign true and false to all of these models. 492 00:25:13,800 --> 00:25:18,210 And we might ask in each one of them, is the knowledge base true? 493 00:25:18,210 --> 00:25:20,130 Here are the set of things that we know. 494 00:25:20,130 --> 00:25:24,420 In which of these worlds could this knowledge base possibly apply to? 495 00:25:24,420 --> 00:25:27,340 In which world is this knowledge base true? 496 00:25:27,340 --> 00:25:29,400 Well, in the knowledge base, for example, we 497 00:25:29,400 --> 00:25:32,680 know P. Like, we know it is a Tuesday. 498 00:25:32,680 --> 00:25:37,260 Which means we know that these four-- first four rows-- where P is false, 499 00:25:37,260 --> 00:25:40,512 none of those are going to be true, are going to work 500 00:25:40,512 --> 00:25:41,970 for this particular knowledge base. 501 00:25:41,970 --> 00:25:45,120 Our knowledge base is not true in those worlds. 502 00:25:45,120 --> 00:25:50,490 Likewise, we also know not Q. We know that it is not raining. 503 00:25:50,490 --> 00:25:54,690 So any of these models where Q is true, like these two and these two 504 00:25:54,690 --> 00:25:59,640 here, those aren't going to work either because we know that Q is not true. 505 00:25:59,640 --> 00:26:03,030 And finally, we also know that P and not Q 506 00:26:03,030 --> 00:26:06,620 implies R. Which means that when P is true-- 507 00:26:06,620 --> 00:26:07,920 where P is true here-- 508 00:26:07,920 --> 00:26:08,970 and Q is false-- 509 00:26:08,970 --> 00:26:10,710 Q is false in these two-- 510 00:26:10,710 --> 00:26:13,110 then R must be true. 511 00:26:13,110 --> 00:26:18,810 And if ever P is true, Q is false, but R is also false, 512 00:26:18,810 --> 00:26:22,050 well, that doesn't satisfy this implication here. 513 00:26:22,050 --> 00:26:26,260 That implication does not hold true under those situations. 514 00:26:26,260 --> 00:26:28,380 So we could say that for our knowledge base, 515 00:26:28,380 --> 00:26:32,280 we can conclude under which of these possible worlds is our knowledge base 516 00:26:32,280 --> 00:26:36,180 true, and under which of the possible worlds is our knowledge base false. 517 00:26:36,180 --> 00:26:39,510 And it turns out there is only one possible world where 518 00:26:39,510 --> 00:26:41,310 our knowledge base is actually true. 519 00:26:41,310 --> 00:26:43,560 In some cases, there might be multiple possible worlds 520 00:26:43,560 --> 00:26:44,935 where the knowledge base is true. 521 00:26:44,935 --> 00:26:47,625 But in this case it just so happens that there's only one. 522 00:26:47,625 --> 00:26:50,100 One possible world where we can definitively 523 00:26:50,100 --> 00:26:52,650 say something about our knowledge base. 524 00:26:52,650 --> 00:26:55,180 And in this case, we would look at the query. 525 00:26:55,180 --> 00:26:57,750 The query of R. Is R true? 526 00:26:57,750 --> 00:26:59,040 R is true. 527 00:26:59,040 --> 00:27:03,110 And so as a result, we can draw that conclusion. 528 00:27:03,110 --> 00:27:05,400 And so this is this idea of model checking. 529 00:27:05,400 --> 00:27:08,880 Enumerate all the possible models, and look in those possible models 530 00:27:08,880 --> 00:27:12,300 to see whether or not if our knowledge base is true, 531 00:27:12,300 --> 00:27:15,658 is the query in question true as well. 532 00:27:15,658 --> 00:27:17,700 So let's now take a look at how we might actually 533 00:27:17,700 --> 00:27:20,640 go about writing this in a programming language like Python. 534 00:27:20,640 --> 00:27:22,560 Take a look at some actual code that would 535 00:27:22,560 --> 00:27:24,810 encode this notion of propositional symbols, 536 00:27:24,810 --> 00:27:29,070 and logic, and these connectives, like and, and or, and not an implication, 537 00:27:29,070 --> 00:27:32,560 and so forth, and see what that code might actually look like. 538 00:27:32,560 --> 00:27:35,097 So I've written in advance a logic library. 539 00:27:35,097 --> 00:27:37,680 It's more detailed than we need to worry about entirely today. 540 00:27:37,680 --> 00:27:40,350 But the important thing is that we have one class 541 00:27:40,350 --> 00:27:45,010 for every type of logical symbol, or connective, that we might have. 542 00:27:45,010 --> 00:27:47,490 So we just have one class for logical symbols, 543 00:27:47,490 --> 00:27:51,030 for example, where every symbol is going to represent and store 544 00:27:51,030 --> 00:27:53,670 some name for that particular symbol. 545 00:27:53,670 --> 00:27:57,240 And we also have a class for not, that takes an operand. 546 00:27:57,240 --> 00:28:00,630 So we might say not one symbol to say something is not true, 547 00:28:00,630 --> 00:28:02,520 or some other sentence is not true. 548 00:28:02,520 --> 00:28:06,450 We have one for and, one for or, so on and so forth. 549 00:28:06,450 --> 00:28:08,170 And I'll just demonstrate how this works. 550 00:28:08,170 --> 00:28:11,740 And you can take a look at the actual logic.py later on. 551 00:28:11,740 --> 00:28:15,140 But go ahead and call this file, Harry.py. 552 00:28:15,140 --> 00:28:19,380 We're going to store information about this world of Harry Potter, 553 00:28:19,380 --> 00:28:20,520 for example. 554 00:28:20,520 --> 00:28:23,430 So I'll go ahead and import from my logic module. 555 00:28:23,430 --> 00:28:24,930 I'll import everything. 556 00:28:24,930 --> 00:28:29,790 And in this library, in order to create a symbol, you use capital S symbol. 557 00:28:29,790 --> 00:28:35,110 And I'll create a symbol for rain to mean it is raining, for example. 558 00:28:35,110 --> 00:28:38,340 And I'll create a symbol for Hagrid to mean 559 00:28:38,340 --> 00:28:41,150 Harry visited Hagrid is what the symbol is going to mean. 560 00:28:41,150 --> 00:28:43,230 So this symbol means it is raining. 561 00:28:43,230 --> 00:28:46,230 This symbol means Harry visited Hagrid. 562 00:28:46,230 --> 00:28:49,830 And I'll add another symbol called Dumbledore 563 00:28:49,830 --> 00:28:54,030 for Harry visited Dumbledore. 564 00:28:54,030 --> 00:28:56,700 Now I'd like to save these symbols so that I can use them later, 565 00:28:56,700 --> 00:28:58,463 as I do some logical analysis. 566 00:28:58,463 --> 00:29:01,130 So I'll go ahead and save each one of them inside of a variable. 567 00:29:01,130 --> 00:29:04,440 So, like, rain, Hagrid, and Dumbledore, that you 568 00:29:04,440 --> 00:29:06,870 could call the variables anything. 569 00:29:06,870 --> 00:29:08,670 And now that I have these logical symbols, 570 00:29:08,670 --> 00:29:12,160 I can use logical connectives to combine them together. 571 00:29:12,160 --> 00:29:19,030 So for example, if I have a sentence like, and rain and Hagrid, 572 00:29:19,030 --> 00:29:19,790 for example-- 573 00:29:19,790 --> 00:29:22,870 which is not necessarily true, but just for demonstration-- 574 00:29:22,870 --> 00:29:26,420 I can now try and print out sentence.formula, 575 00:29:26,420 --> 00:29:29,740 which is a function I wrote that takes a sentence and propositional logic 576 00:29:29,740 --> 00:29:31,870 and just prints it out so that we, the programmers, 577 00:29:31,870 --> 00:29:35,230 can now see this in order to get an understanding for how it actually 578 00:29:35,230 --> 00:29:36,290 works. 579 00:29:36,290 --> 00:29:40,570 So if I run Python Harry.py, what we'll see 580 00:29:40,570 --> 00:29:44,350 is this sentence and propositional logic, rain and Hagrid. 581 00:29:44,350 --> 00:29:46,780 This is the logical representation of what 582 00:29:46,780 --> 00:29:49,180 we have here in our Python program of saying 583 00:29:49,180 --> 00:29:52,360 and, whose arguments are rain and Hagrid. 584 00:29:52,360 --> 00:29:56,130 So we're saying rain and Hagrid by encoding that idea. 585 00:29:56,130 --> 00:29:59,010 And this is quite common in Python object-oriented programming 586 00:29:59,010 --> 00:30:01,100 where you have a number of different classes, 587 00:30:01,100 --> 00:30:04,270 and you pass arguments into them in order to create a new 588 00:30:04,270 --> 00:30:08,110 and object, for example, in order to represent this idea. 589 00:30:08,110 --> 00:30:11,800 But now what I'd like to do is somehow encode the knowledge 590 00:30:11,800 --> 00:30:13,930 that I have about the world, in order to solve 591 00:30:13,930 --> 00:30:15,790 that problem from the beginning of class, 592 00:30:15,790 --> 00:30:18,700 where we talked about trying to figure out who Harry visited, 593 00:30:18,700 --> 00:30:21,550 and trying to figure out if it's raining or if it's not raining. 594 00:30:21,550 --> 00:30:23,830 And so what knowledge do I have? 595 00:30:23,830 --> 00:30:26,990 I'll go ahead and create a new variable called knowledge. 596 00:30:26,990 --> 00:30:27,830 And what do I know? 597 00:30:27,830 --> 00:30:29,663 Well, I know the very first sentence that we 598 00:30:29,663 --> 00:30:33,370 talked about was the idea that if it is not raining, then 599 00:30:33,370 --> 00:30:35,060 Harry will visit Hagrid. 600 00:30:35,060 --> 00:30:35,560 All right. 601 00:30:35,560 --> 00:30:38,020 How do I encode the idea that it is not raining? 602 00:30:38,020 --> 00:30:40,920 Well, I can use not and then the rain symbol. 603 00:30:40,920 --> 00:30:43,510 So here's me saying that it is not raining. 604 00:30:43,510 --> 00:30:49,300 And now the implication is that if it is not raining, then Harry visited Hagrid. 605 00:30:49,300 --> 00:30:52,000 So I'll wrap this inside of an implication 606 00:30:52,000 --> 00:30:56,530 to say, if it is not raining, this first argument to the implication, 607 00:30:56,530 --> 00:31:00,930 well, then Harry visited Hagrid. 608 00:31:00,930 --> 00:31:04,680 So I'm saying implication, the premise is that it's not raining. 609 00:31:04,680 --> 00:31:08,270 And if it is not raining, then Harry visited Hagrid. 610 00:31:08,270 --> 00:31:11,940 And I can print out knowledge.formula to see the logical formula 611 00:31:11,940 --> 00:31:13,990 equivalent of that same idea. 612 00:31:13,990 --> 00:31:17,760 So I run Python of Harry.py, and this is the logical formula 613 00:31:17,760 --> 00:31:20,370 that we see as a result, which is a text-based version of what 614 00:31:20,370 --> 00:31:21,600 we were looking at before. 615 00:31:21,600 --> 00:31:27,540 That if it is not raining, then that implies that Harry visited Hagrid. 616 00:31:27,540 --> 00:31:31,000 But there was additional information that we had access to as well. 617 00:31:31,000 --> 00:31:34,050 In this case, we had access to the fact that Harry 618 00:31:34,050 --> 00:31:37,320 visited either Hagrid or Dumbledore. 619 00:31:37,320 --> 00:31:38,478 So how do I encode that? 620 00:31:38,478 --> 00:31:40,270 Well, this means that in my knowledge, I've 621 00:31:40,270 --> 00:31:42,720 really got multiple pieces of knowledge going on. 622 00:31:42,720 --> 00:31:45,750 I know one thing, and another thing, and another thing. 623 00:31:45,750 --> 00:31:49,475 So I'll go ahead and wrap all of my knowledge inside of an and. 624 00:31:49,475 --> 00:31:51,850 And I'll move things on a new line just for good measure. 625 00:31:51,850 --> 00:31:53,400 But I know multiple things. 626 00:31:53,400 --> 00:31:57,010 So I'm saying knowledge is an and of multiple different sentences. 627 00:31:57,010 --> 00:32:00,190 I know multiple different sentences to be true. 628 00:32:00,190 --> 00:32:03,570 One such sentence that I know to be true is this implication 629 00:32:03,570 --> 00:32:06,900 that if it is not raining, then Harry visited Hagrid. 630 00:32:06,900 --> 00:32:12,960 Another such sentence that I know to be true is or Hagrid Dumbledore. 631 00:32:12,960 --> 00:32:16,710 In other words, so Hagrid or Dumbledore is true, 632 00:32:16,710 --> 00:32:20,580 because I know that Harry visited Hagrid or Dumbledore. 633 00:32:20,580 --> 00:32:22,080 But I know more than that, actually. 634 00:32:22,080 --> 00:32:24,720 That initial sentence from before said that Harry visited 635 00:32:24,720 --> 00:32:27,730 Hagrid or Dumbledore, but not both. 636 00:32:27,730 --> 00:32:30,870 So now I want a sentence that'll encode the idea that Harry didn't 637 00:32:30,870 --> 00:32:34,140 visit both Hagrid and Dumbledore. 638 00:32:34,140 --> 00:32:37,440 Well, the notion of Harry visiting Hagrid and Dumbledore 639 00:32:37,440 --> 00:32:39,000 would be represented like this. 640 00:32:39,000 --> 00:32:42,720 And of Hagrid and Dumbledore. 641 00:32:42,720 --> 00:32:46,290 And if that is not true, I want to say not that, then I'll just 642 00:32:46,290 --> 00:32:50,310 wrap this whole thing inside of a not. 643 00:32:50,310 --> 00:32:54,510 So now these three lines, line 8 says that if it is not raining, then 644 00:32:54,510 --> 00:32:56,040 Harry visited Hagrid. 645 00:32:56,040 --> 00:33:00,060 Line 9 says Harry visited Hagrid or Dumbledore. 646 00:33:00,060 --> 00:33:05,280 And line 10 says Harry didn't visit both Hagrid and Dumbledore. 647 00:33:05,280 --> 00:33:09,510 That it is not true that both the Hagrid symbol and the Dumbledore symbol 648 00:33:09,510 --> 00:33:10,330 are true. 649 00:33:10,330 --> 00:33:12,570 Only one of them can be true. 650 00:33:12,570 --> 00:33:15,690 And finally, the last piece of information that I knew 651 00:33:15,690 --> 00:33:19,620 was the fact that Harry visited Dumbledore. 652 00:33:19,620 --> 00:33:22,440 So these now are the pieces of knowledge that I know. 653 00:33:22,440 --> 00:33:25,730 One sentence, and another sentence, and another, and another. 654 00:33:25,730 --> 00:33:28,920 And I can print out what I know, just to see it a little bit more visually. 655 00:33:28,920 --> 00:33:32,970 And here now is a logical representation of the information 656 00:33:32,970 --> 00:33:35,670 that my computer is now internally representing 657 00:33:35,670 --> 00:33:38,070 using these various different Python objects. 658 00:33:38,070 --> 00:33:41,460 And again, take a look at logic.py if you want to take a look at how exactly 659 00:33:41,460 --> 00:33:42,420 it's implementing this. 660 00:33:42,420 --> 00:33:45,040 But no need to worry too much about all of the details there. 661 00:33:45,040 --> 00:33:49,140 We're here saying that if it is not raining, then Harry visited Hagrid. 662 00:33:49,140 --> 00:33:52,230 We're saying that Hagrid or Dumbledore is true. 663 00:33:52,230 --> 00:33:56,910 And we're saying it is not the case that Hagrid and Dumbledore is true. 664 00:33:56,910 --> 00:33:58,470 That they're not both true. 665 00:33:58,470 --> 00:34:01,720 And we also know that Dumbledore is true. 666 00:34:01,720 --> 00:34:05,550 So this long, logical sentence represents our knowledge base. 667 00:34:05,550 --> 00:34:07,670 It is the thing that we know. 668 00:34:07,670 --> 00:34:12,090 And now what we like to do is we like to use model checking to ask a query. 669 00:34:12,090 --> 00:34:14,639 To ask a question like, based on this information, 670 00:34:14,639 --> 00:34:16,525 do I know whether or not it's raining? 671 00:34:16,525 --> 00:34:18,900 And we, as humans, we're able to logic our way through it 672 00:34:18,900 --> 00:34:22,108 and figure out that, all right, based on these sentences we can conclude this 673 00:34:22,108 --> 00:34:24,750 and that to figure out that yes, it must have been raining. 674 00:34:24,750 --> 00:34:28,090 But now we'd like for the computer to do that as well. 675 00:34:28,090 --> 00:34:30,480 So let's take a look at the model checking algorithm that 676 00:34:30,480 --> 00:34:32,760 is going to follow that same pattern that we drew out 677 00:34:32,760 --> 00:34:34,763 in pseudocode a moment ago. 678 00:34:34,763 --> 00:34:36,555 So I've defined a function here in logic.py 679 00:34:36,555 --> 00:34:40,199 that you can take a look at called model check. 680 00:34:40,199 --> 00:34:43,770 Model check takes two arguments, the knowledge that I already know 681 00:34:43,770 --> 00:34:45,330 and the query. 682 00:34:45,330 --> 00:34:47,730 And the idea is, in order to do model checking, 683 00:34:47,730 --> 00:34:50,440 I need to enumerate all of the possible models. 684 00:34:50,440 --> 00:34:53,580 And for each of the possible models, I need to ask myself, 685 00:34:53,580 --> 00:34:57,110 is the knowledge base true, and is the query true? 686 00:34:57,110 --> 00:34:58,860 So the first thing I need to do is somehow 687 00:34:58,860 --> 00:35:01,050 enumerate all of the possible models. 688 00:35:01,050 --> 00:35:03,750 Meaning for all possible symbols that exist, 689 00:35:03,750 --> 00:35:06,750 I need to assign true and false to each one of them 690 00:35:06,750 --> 00:35:09,312 and see whether or not it's still true. 691 00:35:09,312 --> 00:35:11,820 And so here is the way we're going to do that. 692 00:35:11,820 --> 00:35:13,067 We're going to start-- 693 00:35:13,067 --> 00:35:15,150 so I've defined another helper function internally 694 00:35:15,150 --> 00:35:16,690 that we'll get to in just a moment. 695 00:35:16,690 --> 00:35:20,100 But this function starts by getting all of the symbols, 696 00:35:20,100 --> 00:35:23,008 in both the knowledge and the query, by figuring out 697 00:35:23,008 --> 00:35:24,300 what symbols am I dealing with? 698 00:35:24,300 --> 00:35:26,070 In this case, the symbols I'm dealing with 699 00:35:26,070 --> 00:35:28,410 are rain, and Hagrid, and Dumbledore. 700 00:35:28,410 --> 00:35:30,870 But there might be other symbols, depending on the problem. 701 00:35:30,870 --> 00:35:33,960 And we'll take a look soon at some examples of situations 702 00:35:33,960 --> 00:35:37,170 where ultimately, we're going to need some additional symbols in order 703 00:35:37,170 --> 00:35:39,000 to represent the problem. 704 00:35:39,000 --> 00:35:42,510 And then we're going to run this check all function, which 705 00:35:42,510 --> 00:35:46,260 is a helper function that's basically going to recursively call itself, 706 00:35:46,260 --> 00:35:51,180 checking every possible configuration of propositional symbols. 707 00:35:51,180 --> 00:35:56,560 So we start out by looking at this check all function, and what do we do? 708 00:35:56,560 --> 00:36:01,350 So if not symbols means if we finish assigning all of the symbols. 709 00:36:01,350 --> 00:36:03,120 We've assigned every symbol of value. 710 00:36:03,120 --> 00:36:07,580 So far we haven't done that, but if we ever do, then we check. 711 00:36:07,580 --> 00:36:09,697 In this model, is the knowledge true? 712 00:36:09,697 --> 00:36:11,030 That's what this line is saying. 713 00:36:11,030 --> 00:36:14,300 If we evaluate the knowledge, propositional logic formula, 714 00:36:14,300 --> 00:36:18,590 using the models assignment of truth values, is the knowledge true? 715 00:36:18,590 --> 00:36:23,780 If the knowledge is true, then we should return true, only if the query is true. 716 00:36:23,780 --> 00:36:27,590 Because if the knowledge is true, we want the query to be true as well, 717 00:36:27,590 --> 00:36:29,750 in order for there to be entailment. 718 00:36:29,750 --> 00:36:32,150 Otherwise, we don't know that there-- otherwise, there 719 00:36:32,150 --> 00:36:33,350 won't be an entailment. 720 00:36:33,350 --> 00:36:37,280 If there's ever a situation where what we know in our knowledge is true, 721 00:36:37,280 --> 00:36:40,565 but the query, the thing we're asking, happens to be false. 722 00:36:40,565 --> 00:36:42,440 So this line here is checking that same idea. 723 00:36:42,440 --> 00:36:48,320 That in all worlds where the knowledge is true, the query must also be true. 724 00:36:48,320 --> 00:36:52,010 Otherwise, we can just return true, because if the knowledge isn't true, 725 00:36:52,010 --> 00:36:52,913 then we don't care. 726 00:36:52,913 --> 00:36:54,830 This is equivalent to when we were enumerating 727 00:36:54,830 --> 00:36:56,540 this table from a moment ago. 728 00:36:56,540 --> 00:36:59,540 In all situations where the knowledge base wasn't true-- 729 00:36:59,540 --> 00:37:01,190 all of these seven rows here-- 730 00:37:01,190 --> 00:37:04,460 we didn't care whether or not our query was true or not. 731 00:37:04,460 --> 00:37:07,370 We only care to check whether the query is true 732 00:37:07,370 --> 00:37:09,800 when the knowledge base is actually true, 733 00:37:09,800 --> 00:37:13,130 which was just this green highlighted row right there. 734 00:37:13,130 --> 00:37:16,850 So that logic is encoded using that statement there. 735 00:37:16,850 --> 00:37:19,490 And otherwise, if we haven't assigned symbols yet, 736 00:37:19,490 --> 00:37:22,490 which we haven't seen anything yet, then the first thing we do 737 00:37:22,490 --> 00:37:24,920 is pop one of the symbols. 738 00:37:24,920 --> 00:37:27,800 I make a copy of the symbols first, just to save an existing copy. 739 00:37:27,800 --> 00:37:30,500 But I pop one symbol off of the remaining symbols, 740 00:37:30,500 --> 00:37:33,380 so that I just pick one symbol at random. 741 00:37:33,380 --> 00:37:37,970 And I create one copy of the model where that symbol is true, 742 00:37:37,970 --> 00:37:42,350 and I create a second copy of the model where that symbol is false. 743 00:37:42,350 --> 00:37:44,240 So I now have two copies of the model. 744 00:37:44,240 --> 00:37:47,480 One where the symbol is true, and one where the symbol is false. 745 00:37:47,480 --> 00:37:50,330 And I need to make sure that this entailment holds 746 00:37:50,330 --> 00:37:52,190 in both of those models. 747 00:37:52,190 --> 00:37:56,480 So I recursively check all on the model where the statement is true, 748 00:37:56,480 --> 00:38:01,283 and check all on the model where the statement is false. 749 00:38:01,283 --> 00:38:03,200 So again, you can take a look at that function 750 00:38:03,200 --> 00:38:06,443 to try to get a sense for how exactly this logic is working. 751 00:38:06,443 --> 00:38:09,110 But in effect, what it's doing is recursively calling this check 752 00:38:09,110 --> 00:38:11,330 all function again and again and again. 753 00:38:11,330 --> 00:38:14,090 And on every level of the recursion we're saying, 754 00:38:14,090 --> 00:38:17,600 let's pick a new symbol that we haven't yet assigned. 755 00:38:17,600 --> 00:38:18,830 Assign it to true. 756 00:38:18,830 --> 00:38:20,430 And assign it to false. 757 00:38:20,430 --> 00:38:23,690 And then check to make sure that the entailment holds in both cases. 758 00:38:23,690 --> 00:38:26,480 Because ultimately, I need to check every possible world. 759 00:38:26,480 --> 00:38:28,700 I need to take every combination of symbols 760 00:38:28,700 --> 00:38:31,820 and try every combination of true and false 761 00:38:31,820 --> 00:38:36,040 in order to figure out whether the entailment relation actually holds. 762 00:38:36,040 --> 00:38:38,730 So that function we've written for you. 763 00:38:38,730 --> 00:38:42,560 But in order to use that function inside of Harry.py, what I'll write 764 00:38:42,560 --> 00:38:43,850 is something like this. 765 00:38:43,850 --> 00:38:47,600 I would like to model check, based on the knowledge, 766 00:38:47,600 --> 00:38:50,570 and then I provide, as a second argument, what the query is. 767 00:38:50,570 --> 00:38:52,430 What the thing I want to ask is. 768 00:38:52,430 --> 00:38:56,170 And what I want to ask in this case is, is it raining? 769 00:38:56,170 --> 00:38:58,460 So model check, again, takes two arguments. 770 00:38:58,460 --> 00:39:01,090 The first argument is the information that I know. 771 00:39:01,090 --> 00:39:01,780 This knowledge. 772 00:39:01,780 --> 00:39:06,250 Which in this case, is this information that was given to me at the beginning. 773 00:39:06,250 --> 00:39:11,110 And the second argument, rain, is encoding the idea of the query. 774 00:39:11,110 --> 00:39:12,040 What am I asking? 775 00:39:12,040 --> 00:39:14,410 I would like to ask, based on this knowledge, 776 00:39:14,410 --> 00:39:17,460 do I know for sure that it is raining? 777 00:39:17,460 --> 00:39:21,590 And I can try and print out the result of that. 778 00:39:21,590 --> 00:39:24,930 And when I run this program, I see that the answer is true. 779 00:39:24,930 --> 00:39:29,080 That based on this information, I can conclusively say that it is raining. 780 00:39:29,080 --> 00:39:31,090 Because using this model checking algorithm, 781 00:39:31,090 --> 00:39:35,230 we were able to check that in every world where this knowledge is true, 782 00:39:35,230 --> 00:39:36,010 it is raining. 783 00:39:36,010 --> 00:39:39,550 In other words, there's no world where this knowledge is true 784 00:39:39,550 --> 00:39:41,000 and it is not raining. 785 00:39:41,000 --> 00:39:45,500 So you can conclude that it is, in fact, raining. 786 00:39:45,500 --> 00:39:47,950 And this sort of logic can be applied to a number 787 00:39:47,950 --> 00:39:49,450 of different types of problems. 788 00:39:49,450 --> 00:39:53,170 That if confronted with a problem where some sort of logical deduction 789 00:39:53,170 --> 00:39:55,330 can be used in order to try to solve it, you 790 00:39:55,330 --> 00:39:58,150 might try thinking about what propositional symbols 791 00:39:58,150 --> 00:40:00,760 you might need in order to represent that information. 792 00:40:00,760 --> 00:40:03,220 And what statements and propositional logic 793 00:40:03,220 --> 00:40:07,710 you might use in order to encode that information which you know. 794 00:40:07,710 --> 00:40:10,030 And this process of trying to take a problem 795 00:40:10,030 --> 00:40:12,730 and figure out what propositional symbols to use in order 796 00:40:12,730 --> 00:40:15,850 to encode that idea, or how to represent it logically 797 00:40:15,850 --> 00:40:17,950 is known as knowledge engineering. 798 00:40:17,950 --> 00:40:20,830 That software engineers and AI engineers will take a problem 799 00:40:20,830 --> 00:40:23,980 and try and figure out how to distill it down into knowledge 800 00:40:23,980 --> 00:40:26,847 that is representable by a computer. 801 00:40:26,847 --> 00:40:29,680 And if we can take any general purpose problem, some problem that we 802 00:40:29,680 --> 00:40:32,700 find in the human world, and turn it into a problem 803 00:40:32,700 --> 00:40:35,110 that computer's know how to solve, as by using 804 00:40:35,110 --> 00:40:37,610 any number of different variables, well, then, we 805 00:40:37,610 --> 00:40:39,610 can take a computer that is able to do something 806 00:40:39,610 --> 00:40:42,250 like model checking or some other inference algorithm, 807 00:40:42,250 --> 00:40:46,250 and actually figure out how to solve that problem. 808 00:40:46,250 --> 00:40:49,598 So now we'll take a look at two or three examples of knowledge engineering 809 00:40:49,598 --> 00:40:50,140 and practice. 810 00:40:50,140 --> 00:40:54,460 Of taking some problem and figuring out how we can apply logical symbols 811 00:40:54,460 --> 00:40:58,300 and use logical formulas to be able to encode that idea. 812 00:40:58,300 --> 00:41:01,270 And we'll start with a very popular board game in the US and the UK, 813 00:41:01,270 --> 00:41:02,270 known as Clue. 814 00:41:02,270 --> 00:41:05,170 Now, in the game of Clue, there's a number of different factors 815 00:41:05,170 --> 00:41:06,140 that are going on. 816 00:41:06,140 --> 00:41:08,680 But the basic premise of the game if you've never played it before, 817 00:41:08,680 --> 00:41:10,680 is that there are a number of different people-- 818 00:41:10,680 --> 00:41:13,420 for now we'll just use three, Colonel Mustard, Professor Plum, 819 00:41:13,420 --> 00:41:14,330 and Miss Scarlet. 820 00:41:14,330 --> 00:41:17,080 There are a number of different rooms, like a ballroom, a kitchen, 821 00:41:17,080 --> 00:41:17,870 and a library. 822 00:41:17,870 --> 00:41:20,620 And there are a number of different weapons-- a knife, a revolver, 823 00:41:20,620 --> 00:41:21,700 and a wrench. 824 00:41:21,700 --> 00:41:26,140 And three of these-- one person, one room, and one weapon-- 825 00:41:26,140 --> 00:41:29,500 is the solution to the mystery-- 826 00:41:29,500 --> 00:41:32,788 the murderer and what room they were in, and what weapon they happen to use. 827 00:41:32,788 --> 00:41:35,830 And what happens at the beginning of the game is that all these cards are 828 00:41:35,830 --> 00:41:38,740 randomly shuffled together, and three of them-- one person, one room, 829 00:41:38,740 --> 00:41:39,670 and one weapon-- 830 00:41:39,670 --> 00:41:42,100 are placed into a sealed envelope that we don't know. 831 00:41:42,100 --> 00:41:45,940 And we would like to figure out, using some sort of logical process, what's 832 00:41:45,940 --> 00:41:46,870 inside the envelope. 833 00:41:46,870 --> 00:41:49,840 Which person, which room, and which weapon. 834 00:41:49,840 --> 00:41:54,340 And we do so by looking at some, but not all, of these cards here. 835 00:41:54,340 --> 00:41:59,198 By looking at these cards to try and figure out what might be going on. 836 00:41:59,198 --> 00:42:01,240 And so this is a very popular game, but let's now 837 00:42:01,240 --> 00:42:04,120 try and formalize it and see if we could train a computer 838 00:42:04,120 --> 00:42:08,410 to be able to play this game by reasoning through it logically. 839 00:42:08,410 --> 00:42:10,540 So in order to do this we'll begin by thinking 840 00:42:10,540 --> 00:42:13,780 about what propositional symbols we're ultimately going to need. 841 00:42:13,780 --> 00:42:16,240 Remember again, that propositional symbols are just 842 00:42:16,240 --> 00:42:21,950 some symbol, some variable, that can be either true or false in the world. 843 00:42:21,950 --> 00:42:24,340 And so in this case, the propositional symbols 844 00:42:24,340 --> 00:42:29,290 are really just going to correspond to each of the possible things that 845 00:42:29,290 --> 00:42:30,730 could be inside the envelope. 846 00:42:30,730 --> 00:42:33,740 Mustard is a propositional symbol, that in this case 847 00:42:33,740 --> 00:42:37,010 will just be true, if Colonel Mustard is inside the envelope, 848 00:42:37,010 --> 00:42:38,200 if he is the murderer. 849 00:42:38,200 --> 00:42:39,460 And false otherwise. 850 00:42:39,460 --> 00:42:42,790 And likewise, for Plum for Professor Plum, and Scarlet for Miss Scarlet, 851 00:42:42,790 --> 00:42:45,880 and likewise for each of the rooms, and for each of the weapons. 852 00:42:45,880 --> 00:42:50,450 We have one propositional symbol for each of these ideas. 853 00:42:50,450 --> 00:42:52,870 Then using those propositional symbols we 854 00:42:52,870 --> 00:42:56,620 can begin to create logical sentences, create knowledge 855 00:42:56,620 --> 00:42:58,640 that we know about the world. 856 00:42:58,640 --> 00:43:01,810 So for example, we know that someone is the murderer. 857 00:43:01,810 --> 00:43:04,840 That one of the three people is, in fact, the murderer. 858 00:43:04,840 --> 00:43:06,220 And how would we encode that? 859 00:43:06,220 --> 00:43:08,590 Well, we don't know for sure who the murderer is, 860 00:43:08,590 --> 00:43:13,730 but we know it is one person, or the second person, or the third person. 861 00:43:13,730 --> 00:43:18,040 So I could say something like this: Mustard, or Plum, or Scarlet. 862 00:43:18,040 --> 00:43:21,573 And this piece of knowledge encodes that one of these three people 863 00:43:21,573 --> 00:43:22,240 is the murderer. 864 00:43:22,240 --> 00:43:26,980 We don't know which, but one of these three things must be true. 865 00:43:26,980 --> 00:43:28,700 What other information do we know? 866 00:43:28,700 --> 00:43:30,893 Well, we know that, for example, one of the rooms 867 00:43:30,893 --> 00:43:32,560 must have been the room in the envelope. 868 00:43:32,560 --> 00:43:36,460 That the crime was committed either in the ballroom, or the kitchen, 869 00:43:36,460 --> 00:43:37,363 or the library. 870 00:43:37,363 --> 00:43:39,280 Again, right now we don't know which, but this 871 00:43:39,280 --> 00:43:42,640 is knowledge we know at the outset-- knowledge that one of these three 872 00:43:42,640 --> 00:43:44,417 must be inside the envelope. 873 00:43:44,417 --> 00:43:46,750 And likewise we can say the same thing about the weapon. 874 00:43:46,750 --> 00:43:49,750 That it was either the knife, or the revolver, or the wrench. 875 00:43:49,750 --> 00:43:52,780 That one of those weapons must have been the weapon of choice, 876 00:43:52,780 --> 00:43:55,950 and therefore, the weapon in the envelope. 877 00:43:55,950 --> 00:43:57,960 And then as the game progresses, the game play 878 00:43:57,960 --> 00:44:00,120 works by people get various different cards. 879 00:44:00,120 --> 00:44:03,200 And using those cards, you can deduce information. 880 00:44:03,200 --> 00:44:05,380 That if someone gives you a card, for example, 881 00:44:05,380 --> 00:44:08,490 I have the Professor Plum card in my hand, 882 00:44:08,490 --> 00:44:12,240 then I know the Professor Plum card can't be inside the envelope. 883 00:44:12,240 --> 00:44:15,570 I know that Professor Plum is not the criminal. 884 00:44:15,570 --> 00:44:19,320 So I know a piece of information, like not Plum, for example. 885 00:44:19,320 --> 00:44:22,710 I know that Professor Plum has to be false. 886 00:44:22,710 --> 00:44:25,650 This propositional symbol is not true. 887 00:44:25,650 --> 00:44:29,040 And sometimes I might not know for sure that a particular card is not 888 00:44:29,040 --> 00:44:29,672 in the middle. 889 00:44:29,672 --> 00:44:31,380 But sometimes, someone will make a guess, 890 00:44:31,380 --> 00:44:34,510 and I'll know that one of three possibilities is not true. 891 00:44:34,510 --> 00:44:37,890 Like someone will guess Colonel Mustard in the library with the revolver, 892 00:44:37,890 --> 00:44:39,370 or something to that effect. 893 00:44:39,370 --> 00:44:42,360 And in that case, a card might be revealed that I don't see. 894 00:44:42,360 --> 00:44:47,340 But if it is a card, and it is either Colonel Mustard, or the revolver, 895 00:44:47,340 --> 00:44:51,002 or the library, then I know that at least one of them 896 00:44:51,002 --> 00:44:51,960 can't be in the middle. 897 00:44:51,960 --> 00:44:57,600 So I know something like, it is either not Mustard, or it is not the library, 898 00:44:57,600 --> 00:44:59,670 or it is not the revolver. 899 00:44:59,670 --> 00:45:03,000 Now maybe multiple of these are not true, but I know that at least one 900 00:45:03,000 --> 00:45:08,330 of Mustard, library and revolver must, in fact, be false. 901 00:45:08,330 --> 00:45:13,080 And so this now is a propositional logic representation of this game of Clue. 902 00:45:13,080 --> 00:45:16,560 A way of encoding the knowledge that we know inside this game using 903 00:45:16,560 --> 00:45:19,380 propositional logic that a computer algorithm, something 904 00:45:19,380 --> 00:45:21,600 like model checking that we saw a moment ago, 905 00:45:21,600 --> 00:45:24,280 can actually look at and understand. 906 00:45:24,280 --> 00:45:28,200 So let's now take a look at some code to see how this algorithm might actually 907 00:45:28,200 --> 00:45:31,020 work in practice. 908 00:45:31,020 --> 00:45:31,520 All right. 909 00:45:31,520 --> 00:45:33,905 So I'm now going to open up a file called clue.py, 910 00:45:33,905 --> 00:45:35,272 which I've started already. 911 00:45:35,272 --> 00:45:37,730 And what we'll see here is I've defined a couple of things. 912 00:45:37,730 --> 00:45:39,680 I've defined some symbols initially. 913 00:45:39,680 --> 00:45:42,980 Notice I have a symbol for Colonel Mustard, a symbol for Professor Plum, 914 00:45:42,980 --> 00:45:45,290 a symbol for Miss Scarlet, all of which I put inside 915 00:45:45,290 --> 00:45:46,910 of this list of characters. 916 00:45:46,910 --> 00:45:48,710 I have a symbol for ballroom, and kitchen, 917 00:45:48,710 --> 00:45:50,950 and library inside of a list of rooms. 918 00:45:50,950 --> 00:45:53,720 And then I have symbols for knife, and revolver, and wrench. 919 00:45:53,720 --> 00:45:55,070 These are my weapons. 920 00:45:55,070 --> 00:45:58,070 And so all of these characters and rooms and weapons altogether, 921 00:45:58,070 --> 00:46:00,140 those are my symbols. 922 00:46:00,140 --> 00:46:03,600 And now I also have this check knowledge function. 923 00:46:03,600 --> 00:46:07,070 And what the check knowledge function does is it takes my knowledge, 924 00:46:07,070 --> 00:46:11,600 and it's going to try and draw conclusions about what I know. 925 00:46:11,600 --> 00:46:16,100 So for example, we'll loop over all of the possible symbols and we'll check. 926 00:46:16,100 --> 00:46:17,750 Do I know that that symbol is true? 927 00:46:17,750 --> 00:46:20,750 And a symbol is going to be something like Professor Plum, or the knife, 928 00:46:20,750 --> 00:46:21,780 or the library. 929 00:46:21,780 --> 00:46:23,820 And if I know that it is true, in other words, 930 00:46:23,820 --> 00:46:26,660 I know that it must be the card in the envelope, 931 00:46:26,660 --> 00:46:29,180 then I'm going to print out, using a function called 932 00:46:29,180 --> 00:46:30,980 C print, which prints things in color. 933 00:46:30,980 --> 00:46:34,063 I'm going to print out the word yes, and I'm going to print that in green, 934 00:46:34,063 --> 00:46:36,520 just to make it very clear to us. 935 00:46:36,520 --> 00:46:39,450 And if we're not sure that the symbol is true, 936 00:46:39,450 --> 00:46:42,930 maybe I can check to see if I'm sure that the symbol is not true. 937 00:46:42,930 --> 00:46:46,830 Like, if I know for sure that it is not Professor Plum, for example. 938 00:46:46,830 --> 00:46:49,110 And I do that by running model check again. 939 00:46:49,110 --> 00:46:52,690 This time checking if my knowledge is not the symbol. 940 00:46:52,690 --> 00:46:56,460 If I know for sure that the symbol is not true. 941 00:46:56,460 --> 00:46:59,430 And if I don't know for sure that the symbol is not true, 942 00:46:59,430 --> 00:47:03,140 because I say elif not model check, meaning I'm not sure 943 00:47:03,140 --> 00:47:04,980 that the symbol is false, well, then I'll 944 00:47:04,980 --> 00:47:07,350 go ahead and print out maybe next to the symbol. 945 00:47:07,350 --> 00:47:09,060 Because maybe the symbol is true. 946 00:47:09,060 --> 00:47:10,140 Maybe it's not. 947 00:47:10,140 --> 00:47:12,180 I don't actually know. 948 00:47:12,180 --> 00:47:14,550 So what knowledge do I actually have? 949 00:47:14,550 --> 00:47:16,650 Well, let's try and represent my knowledge now. 950 00:47:16,650 --> 00:47:20,730 So my knowledge is, I know a couple of things so I'll put them in an and. 951 00:47:20,730 --> 00:47:24,570 And I know that one of the three people must be the criminal. 952 00:47:24,570 --> 00:47:28,117 So I know or Mustard, Plum, Scarlet. 953 00:47:28,117 --> 00:47:31,200 This is my way of encoding that it is either Colonel Mustard, or Professor 954 00:47:31,200 --> 00:47:32,940 Plum, or Miss Scarlet. 955 00:47:32,940 --> 00:47:35,310 I know that it must have happened in one of the rooms. 956 00:47:35,310 --> 00:47:40,320 So I know or ballroom, kitchen, library, for example. 957 00:47:40,320 --> 00:47:43,080 And I know that one of the weapons must have been used as well. 958 00:47:43,080 --> 00:47:47,610 So I know or knife, revolver, wrench. 959 00:47:47,610 --> 00:47:49,380 So that might be my initial knowledge. 960 00:47:49,380 --> 00:47:51,400 That I know that it must have been one of the people. 961 00:47:51,400 --> 00:47:53,275 I know it must have been in one of the rooms. 962 00:47:53,275 --> 00:47:55,993 And I know that it must have been one of the weapons. 963 00:47:55,993 --> 00:47:58,410 And I can see what that knowledge looks like as a formula, 964 00:47:58,410 --> 00:48:01,100 by printing out knowledge.formula. 965 00:48:01,100 --> 00:48:03,240 So I'll run Python clue.py. 966 00:48:03,240 --> 00:48:06,720 And here now is the information that I know in logical format. 967 00:48:06,720 --> 00:48:10,080 I know that it is Colonel Mustard, or Professor Plum, or Miss Scarlet. 968 00:48:10,080 --> 00:48:13,020 And I know that it is the ballroom, the kitchen, or the library. 969 00:48:13,020 --> 00:48:16,020 And I know that it is the knife, the revolver, or the wrench. 970 00:48:16,020 --> 00:48:18,160 But I don't know much more than that. 971 00:48:18,160 --> 00:48:20,520 I can't really draw any firm conclusions. 972 00:48:20,520 --> 00:48:23,620 And in fact, we can see that if I try and do-- 973 00:48:23,620 --> 00:48:27,962 let me go ahead and run my knowledge check function on my knowledge. 974 00:48:27,962 --> 00:48:29,670 Now let's check is this function that I-- 975 00:48:29,670 --> 00:48:32,170 or check knowledge rather. 976 00:48:32,170 --> 00:48:35,530 Is this function that I just wrote that looks over all of the symbols 977 00:48:35,530 --> 00:48:40,480 and tries to see what conclusions I can actually draw about any of the symbols. 978 00:48:40,480 --> 00:48:45,797 So I'll go ahead and run clue.py and see what it is that I know. 979 00:48:45,797 --> 00:48:48,130 And it seems that I don't really know anything for sure. 980 00:48:48,130 --> 00:48:49,960 I have all three people are maybes. 981 00:48:49,960 --> 00:48:51,520 All three of the rooms are maybes. 982 00:48:51,520 --> 00:48:53,140 All three of the weapons are maybes. 983 00:48:53,140 --> 00:48:56,410 I don't really know anything for certain just yet. 984 00:48:56,410 --> 00:48:59,050 But now let me try and add some additional information 985 00:48:59,050 --> 00:49:01,690 and see if additional information, additional knowledge, 986 00:49:01,690 --> 00:49:04,818 can help us to logically reason our way through this process. 987 00:49:04,818 --> 00:49:06,860 And we are just going to provide the information. 988 00:49:06,860 --> 00:49:11,020 Our AI is going to take care of doing the inference and figuring out 989 00:49:11,020 --> 00:49:13,390 what conclusions it's able to draw. 990 00:49:13,390 --> 00:49:17,030 So I start with some cards, and those cards tell me something. 991 00:49:17,030 --> 00:49:19,900 So if I have the Colonel Mustard card, for example, 992 00:49:19,900 --> 00:49:23,770 I know that the Mustard symbol must be false. 993 00:49:23,770 --> 00:49:26,800 In other words, Mustard is not the one in the envelope. 994 00:49:26,800 --> 00:49:27,970 It's not the criminal. 995 00:49:27,970 --> 00:49:31,150 So I can say, knowledge supports something called-- 996 00:49:31,150 --> 00:49:36,010 every and in this library supports .add, which is a way of adding knowledge, 997 00:49:36,010 --> 00:49:39,890 or adding an additional logical sentence to an and clause. 998 00:49:39,890 --> 00:49:44,590 So I can say, knowledge.add, not Mustard. 999 00:49:44,590 --> 00:49:47,200 I happen to know, because I have the Mustard card, 1000 00:49:47,200 --> 00:49:49,240 that Colonel Mustard is not the suspect. 1001 00:49:49,240 --> 00:49:51,160 And maybe I have a couple of other cards, too. 1002 00:49:51,160 --> 00:49:53,500 Maybe I also have a card for the kitchen, 1003 00:49:53,500 --> 00:49:55,060 so I know it's not the kitchen. 1004 00:49:55,060 --> 00:49:58,752 And maybe I have another card that says that it is not the revolver. 1005 00:49:58,752 --> 00:49:59,710 So I have three cards-- 1006 00:49:59,710 --> 00:50:01,750 Colonel Mustard, the kitchen, and the revolver. 1007 00:50:01,750 --> 00:50:06,190 And I encode that into my AI this way, by saying it's not Colonel Mustard, 1008 00:50:06,190 --> 00:50:10,630 it's not the kitchen, and it's not the revolver, and I know those to be true. 1009 00:50:10,630 --> 00:50:14,080 So now when I rerun clue.py, we'll see that I've been 1010 00:50:14,080 --> 00:50:16,540 able to eliminate some possibilities. 1011 00:50:16,540 --> 00:50:20,410 Before I wasn't sure if it was the knife, or the revolver, or the wrench. 1012 00:50:20,410 --> 00:50:23,080 Knife was maybe, revolver was maybe, wrench was maybe. 1013 00:50:23,080 --> 00:50:25,492 Now I'm down to just the knife and the wrench. 1014 00:50:25,492 --> 00:50:28,450 Between those two, I don't know which one it is-- they're both maybes-- 1015 00:50:28,450 --> 00:50:31,660 but I've been able to eliminate the revolver, which is one 1016 00:50:31,660 --> 00:50:36,160 that I know to be false because I have the revolver card. 1017 00:50:36,160 --> 00:50:40,390 And so additional information might be acquired over the course of this game, 1018 00:50:40,390 --> 00:50:43,120 and we would represent that just by adding knowledge 1019 00:50:43,120 --> 00:50:47,620 to our knowledge set, or knowledge base that we've been building here. 1020 00:50:47,620 --> 00:50:50,347 So if, for example, we additionally got the information 1021 00:50:50,347 --> 00:50:51,430 that someone made a guess. 1022 00:50:51,430 --> 00:50:55,300 Someone guessed, like, Miss Scarlet in the library with the wrench. 1023 00:50:55,300 --> 00:50:59,380 And we know that that card was revealed, which means that one of those three 1024 00:50:59,380 --> 00:51:02,620 cards-- either Miss Scarlet, or the library, or the wrench-- 1025 00:51:02,620 --> 00:51:06,800 one of those at minimum, must not be inside of the envelope. 1026 00:51:06,800 --> 00:51:08,070 So I could add some knowledge. 1027 00:51:08,070 --> 00:51:12,165 Say knowledge.add, and I'm going to add an or clause, because I 1028 00:51:12,165 --> 00:51:13,790 don't know for sure which one it's not. 1029 00:51:13,790 --> 00:51:16,390 But I know one of them is not in the envelope. 1030 00:51:16,390 --> 00:51:19,762 So it's either not Scarlet or it's not the library. 1031 00:51:19,762 --> 00:51:21,220 And or supports multiple arguments. 1032 00:51:21,220 --> 00:51:25,010 I can say it's also or not the wrench. 1033 00:51:25,010 --> 00:51:26,380 So at least one of those-- 1034 00:51:26,380 --> 00:51:29,395 Scarlet, library, and wrench-- at least one of those needs to be false. 1035 00:51:29,395 --> 00:51:30,520 I don't know which, though. 1036 00:51:30,520 --> 00:51:34,930 Maybe it's multiple, maybe it's just one, but at least one I know 1037 00:51:34,930 --> 00:51:36,640 needs to hold. 1038 00:51:36,640 --> 00:51:39,430 And so now if I rerun clue.py, I don't actually 1039 00:51:39,430 --> 00:51:41,860 have any additional information just yet. 1040 00:51:41,860 --> 00:51:43,180 Nothing I can say conclusively. 1041 00:51:43,180 --> 00:51:46,240 I still know that maybe it's Professor Plum, maybe it's Miss Scarlet. 1042 00:51:46,240 --> 00:51:48,705 I haven't eliminated any options. 1043 00:51:48,705 --> 00:51:50,830 But let's imagine that I get some more information. 1044 00:51:50,830 --> 00:51:54,650 That someone shows me the Professor Plum card, for example. 1045 00:51:54,650 --> 00:51:55,600 So I say, all right. 1046 00:51:55,600 --> 00:51:56,740 Let's go back here. 1047 00:51:56,740 --> 00:52:01,323 Knowledge.add, not Plum. 1048 00:52:01,323 --> 00:52:02,740 So I have the Professor Plum card. 1049 00:52:02,740 --> 00:52:04,900 I know that Professor Plum is not in the middle. 1050 00:52:04,900 --> 00:52:09,350 I rerun clue.py, and right now, I'm able to draw some conclusions. 1051 00:52:09,350 --> 00:52:11,440 Now I've been able to eliminate Professor Plum. 1052 00:52:11,440 --> 00:52:14,650 And the only person that could remaining be is Miss Scarlet. 1053 00:52:14,650 --> 00:52:18,610 So I know, yes, Miss Scarlet, this variable must be true. 1054 00:52:18,610 --> 00:52:21,910 And I've been able to infer that based on the information I already had. 1055 00:52:21,910 --> 00:52:24,910 Now, between the ballroom and the library, and the knife and the wrench, 1056 00:52:24,910 --> 00:52:26,860 for those two, I'm still not sure. 1057 00:52:26,860 --> 00:52:29,470 So let's add one more piece of information. 1058 00:52:29,470 --> 00:52:32,230 Let's say that I know that it's not the ballroom. 1059 00:52:32,230 --> 00:52:35,230 Someone has shown me the ballroom card, so I know it's not the ballroom. 1060 00:52:35,230 --> 00:52:38,480 Which means at this point, I should be able to conclude that it's the library. 1061 00:52:38,480 --> 00:52:39,310 Let's see. 1062 00:52:39,310 --> 00:52:44,240 I'll say knowledge.add, not the ballroom. 1063 00:52:44,240 --> 00:52:47,202 And we'll go ahead and run that. 1064 00:52:47,202 --> 00:52:49,410 And it turns out that after all of this, not only can 1065 00:52:49,410 --> 00:52:51,900 I conclude that I know that it's the library. 1066 00:52:51,900 --> 00:52:53,825 But I also know that the weapon was the knife. 1067 00:52:53,825 --> 00:52:56,700 And that might have been an inference that was a little bit trickier. 1068 00:52:56,700 --> 00:52:58,710 Something I wouldn't have realized immediately. 1069 00:52:58,710 --> 00:53:01,247 But the AI, via this model checking algorithm, 1070 00:53:01,247 --> 00:53:02,580 is able to draw that conclusion. 1071 00:53:02,580 --> 00:53:04,410 That we know for sure that it must be Miss 1072 00:53:04,410 --> 00:53:06,690 Scarlet in the library with the knife. 1073 00:53:06,690 --> 00:53:08,200 And how do we know that? 1074 00:53:08,200 --> 00:53:11,760 Well, we know it from this or clause up here. 1075 00:53:11,760 --> 00:53:15,810 That we know that it's either not Scarlet, or it's not the library, 1076 00:53:15,810 --> 00:53:17,730 or it's not the wrench. 1077 00:53:17,730 --> 00:53:20,310 And given that we know that it is Miss Scarlet, 1078 00:53:20,310 --> 00:53:23,280 and we know that it is the library, then the only remaining 1079 00:53:23,280 --> 00:53:26,520 option for the weapon is that it is not the wrench, which 1080 00:53:26,520 --> 00:53:28,620 means that it must be the knife. 1081 00:53:28,620 --> 00:53:31,050 So we, as humans, now can go back and reason through that, 1082 00:53:31,050 --> 00:53:33,258 even though it might not have been immediately clear. 1083 00:53:33,258 --> 00:53:37,112 And that's one of the advantages of using an AI or some sort of algorithm 1084 00:53:37,112 --> 00:53:38,820 in order to do this, is that the computer 1085 00:53:38,820 --> 00:53:42,000 can exhaust all of these possibilities and try and figure out 1086 00:53:42,000 --> 00:53:45,010 what the solution actually should be. 1087 00:53:45,010 --> 00:53:47,550 And so for that reason, it's often helpful to be 1088 00:53:47,550 --> 00:53:49,380 able to represent knowledge in this way. 1089 00:53:49,380 --> 00:53:51,570 Knowledge engineering, some situation, where 1090 00:53:51,570 --> 00:53:54,870 we can use a computer to be able to represent knowledge and draw 1091 00:53:54,870 --> 00:53:57,060 conclusions based on that knowledge. 1092 00:53:57,060 --> 00:54:00,390 And anytime we can translate something into propositional logic 1093 00:54:00,390 --> 00:54:03,690 symbols like this, type of approach can be useful. 1094 00:54:03,690 --> 00:54:05,820 So you might be familiar with logic puzzles, where 1095 00:54:05,820 --> 00:54:08,612 you have to puzzle your way through trying to figure something out. 1096 00:54:08,612 --> 00:54:10,830 This is what a classic logic puzzle might look like. 1097 00:54:10,830 --> 00:54:13,950 Something like Gildaroy, Minerva, Pomona, and Horace each 1098 00:54:13,950 --> 00:54:16,740 belong to a different one of the four houses-- 1099 00:54:16,740 --> 00:54:19,410 Gryffindor, Hufflepuff, Ravenclaw, and Slytherin. 1100 00:54:19,410 --> 00:54:21,750 And then we have some information, that Gildaroy 1101 00:54:21,750 --> 00:54:23,970 belongs to Gryffindor or Ravenclaw. 1102 00:54:23,970 --> 00:54:26,190 Pomona does not belong in Slytherin. 1103 00:54:26,190 --> 00:54:28,740 And Minerva does belong to Gryffindor. 1104 00:54:28,740 --> 00:54:30,510 We have a couple of pieces of information. 1105 00:54:30,510 --> 00:54:32,520 And using that information, we need to be 1106 00:54:32,520 --> 00:54:35,340 able to draw some conclusions about which person 1107 00:54:35,340 --> 00:54:37,590 should be assigned to which house. 1108 00:54:37,590 --> 00:54:41,910 And again, we can use the exact same idea to try and implement this notion. 1109 00:54:41,910 --> 00:54:44,018 So we need some propositional symbols. 1110 00:54:44,018 --> 00:54:45,810 And in this case, the propositional symbols 1111 00:54:45,810 --> 00:54:48,120 are going to get a little more complex, although, we'll 1112 00:54:48,120 --> 00:54:50,820 see ways to make this a little bit cleaner later on. 1113 00:54:50,820 --> 00:54:53,670 But we'll need 16 propositional symbols. 1114 00:54:53,670 --> 00:54:56,280 One for each person and house. 1115 00:54:56,280 --> 00:54:58,890 So we need to say-- remember, every propositional symbol 1116 00:54:58,890 --> 00:55:00,910 is either true or false. 1117 00:55:00,910 --> 00:55:03,600 So Gildaroy Gryffindor is either true or false. 1118 00:55:03,600 --> 00:55:05,820 Either he's in Gryffindor or he is not. 1119 00:55:05,820 --> 00:55:08,070 Likewise, Gildaroy Hufflepuff also true or false. 1120 00:55:08,070 --> 00:55:10,230 Either it is true or it's false. 1121 00:55:10,230 --> 00:55:14,055 And that's true for every combination of person and house 1122 00:55:14,055 --> 00:55:15,180 that we could come up with. 1123 00:55:15,180 --> 00:55:19,200 We have some sort of propositional symbol for each one of those. 1124 00:55:19,200 --> 00:55:21,510 Using this type of knowledge, we can then 1125 00:55:21,510 --> 00:55:24,870 begin to think about what types of logical sentences 1126 00:55:24,870 --> 00:55:26,730 we can say about the puzzle. 1127 00:55:26,730 --> 00:55:28,020 That if we know-- 1128 00:55:28,020 --> 00:55:30,363 before even think about the information we were given, 1129 00:55:30,363 --> 00:55:32,280 we can think about the premise of the problem. 1130 00:55:32,280 --> 00:55:35,850 That every person is assigned to a different house. 1131 00:55:35,850 --> 00:55:37,150 So what does that tell us? 1132 00:55:37,150 --> 00:55:38,760 Well, it tells us sentences like this. 1133 00:55:38,760 --> 00:55:44,190 It tells us, like, Pomona Slytherin implies not Pomona Hufflepuff. 1134 00:55:44,190 --> 00:55:46,530 Something like if Pomona is in Slytherin, 1135 00:55:46,530 --> 00:55:48,990 then we know that Pomona is not in Hufflepuff. 1136 00:55:48,990 --> 00:55:52,800 And we know this for all four people and for all combinations of houses. 1137 00:55:52,800 --> 00:55:55,620 That no matter what you person you pick, if they're in one house, 1138 00:55:55,620 --> 00:55:57,900 then they're not in some other house. 1139 00:55:57,900 --> 00:56:00,390 So I'll probably have a whole bunch of knowledge statements 1140 00:56:00,390 --> 00:56:01,560 that are of this four. 1141 00:56:01,560 --> 00:56:03,480 That if we know Pomona's in Slytherin, then 1142 00:56:03,480 --> 00:56:06,060 we know Pomona is not in Hufflepuff. 1143 00:56:06,060 --> 00:56:10,110 We were also given the information that each person is in a different house. 1144 00:56:10,110 --> 00:56:12,840 So I also have pieces of knowledge that look something like this. 1145 00:56:12,840 --> 00:56:17,490 Minerva Ravenclaw implies not Gildaroy Ravenclaw. 1146 00:56:17,490 --> 00:56:20,880 If they're all in different houses, then if Minerva is in Ravenclaw, 1147 00:56:20,880 --> 00:56:24,330 then we know that Gildaroy is not in Ravenclaw as well. 1148 00:56:24,330 --> 00:56:26,310 And I have a whole bunch of similar sentences 1149 00:56:26,310 --> 00:56:30,600 like this that are expressing that idea for other people and other houses 1150 00:56:30,600 --> 00:56:31,475 as well. 1151 00:56:31,475 --> 00:56:34,150 And so in addition to sentences of these form, 1152 00:56:34,150 --> 00:56:36,420 I also have the knowledge that was given to me. 1153 00:56:36,420 --> 00:56:40,200 Information like, Gildaroy was in Gryffindor or in Ravenclaw 1154 00:56:40,200 --> 00:56:41,730 that would be represented like this. 1155 00:56:41,730 --> 00:56:44,970 Gildaroy Gryffindor or Gildaroy Ravenclaw. 1156 00:56:44,970 --> 00:56:47,310 And then using these sorts of sentences, I 1157 00:56:47,310 --> 00:56:51,077 can begin to draw some conclusions about the world. 1158 00:56:51,077 --> 00:56:52,410 So let's see an example of this. 1159 00:56:52,410 --> 00:56:55,110 We'll go ahead and actually try and implement this logic puzzle 1160 00:56:55,110 --> 00:56:57,660 to see if we can figure out what the answer is. 1161 00:56:57,660 --> 00:57:00,960 I'll go ahead and open up puzzle.py where I've already 1162 00:57:00,960 --> 00:57:03,150 started to implement this sort of idea. 1163 00:57:03,150 --> 00:57:06,180 I've defined a list of people and a list of houses. 1164 00:57:06,180 --> 00:57:11,057 And I've, so far, created one symbol for every person and for every house. 1165 00:57:11,057 --> 00:57:13,890 That's what this double for loop is doing-- looping over all people, 1166 00:57:13,890 --> 00:57:17,850 looping over all houses, creating a new symbol for each of them. 1167 00:57:17,850 --> 00:57:20,620 And then I've added some information. 1168 00:57:20,620 --> 00:57:23,640 I know that every person belongs to a house, 1169 00:57:23,640 --> 00:57:26,880 so I've added the information for every person-- 1170 00:57:26,880 --> 00:57:29,970 that person Gryffindor, or person Hufflepuff, 1171 00:57:29,970 --> 00:57:32,580 or person Ravenclaw, or person Slytherin. 1172 00:57:32,580 --> 00:57:35,130 That one of those four things must be true. 1173 00:57:35,130 --> 00:57:37,530 Every person belongs to a house. 1174 00:57:37,530 --> 00:57:39,180 What other information do I know? 1175 00:57:39,180 --> 00:57:42,390 I also know that only one house per person. 1176 00:57:42,390 --> 00:57:45,990 So no person belongs to multiple houses. 1177 00:57:45,990 --> 00:57:47,113 So how does this work? 1178 00:57:47,113 --> 00:57:49,030 Well, this is going to be true for all people. 1179 00:57:49,030 --> 00:57:51,390 So I'll loop over every person. 1180 00:57:51,390 --> 00:57:55,380 And then I need to loop over all different pairs of houses. 1181 00:57:55,380 --> 00:57:59,130 The idea is I want to encode the idea that if Minerva is in Gryffindor, 1182 00:57:59,130 --> 00:58:01,770 then Minerva can't be in Ravenclaw. 1183 00:58:01,770 --> 00:58:04,050 So I'll loop over all houses, h1. 1184 00:58:04,050 --> 00:58:06,870 And I'll loop over all houses again, h2. 1185 00:58:06,870 --> 00:58:10,500 And as long as they're different, h1 not equal to H2, 1186 00:58:10,500 --> 00:58:13,500 then I'll add to my knowledge base this piece of information. 1187 00:58:13,500 --> 00:58:18,600 That implication, in other words, an if/then, if the person is in h1, 1188 00:58:18,600 --> 00:58:22,860 then I know that they are not in house h2. 1189 00:58:22,860 --> 00:58:25,170 So these lines here are encoding the notion 1190 00:58:25,170 --> 00:58:28,050 that for every person, if they belong to house one, then 1191 00:58:28,050 --> 00:58:30,273 they are not in house two. 1192 00:58:30,273 --> 00:58:32,190 And the other piece of logic we need to encode 1193 00:58:32,190 --> 00:58:35,290 is the idea that every house can only have one person. 1194 00:58:35,290 --> 00:58:38,370 In other words, if Pomona is Hufflepuff, then nobody else 1195 00:58:38,370 --> 00:58:40,260 is allowed to be in Hufflepuff either. 1196 00:58:40,260 --> 00:58:42,260 And that's the same logic but sort of backwards. 1197 00:58:42,260 --> 00:58:47,040 I loop over all of the houses, and loop over all different pairs of people. 1198 00:58:47,040 --> 00:58:49,890 So I loop over people once, loop over people again. 1199 00:58:49,890 --> 00:58:54,390 And only do this when the people are different, p1 not equal to p2. 1200 00:58:54,390 --> 00:58:59,250 And I add the knowledge that if, as given by the implication, 1201 00:58:59,250 --> 00:59:02,640 if person one belongs to the house, then it 1202 00:59:02,640 --> 00:59:08,247 is not the case that person two belongs to the same house. 1203 00:59:08,247 --> 00:59:10,080 So here I'm just encoding the knowledge that 1204 00:59:10,080 --> 00:59:12,150 represents the problems constraints. 1205 00:59:12,150 --> 00:59:14,040 I know that everyone's in a different house. 1206 00:59:14,040 --> 00:59:17,070 I know that any person can only belong to one house. 1207 00:59:17,070 --> 00:59:21,750 And I can now take my knowledge and try and print out the information 1208 00:59:21,750 --> 00:59:22,890 that I happen to know. 1209 00:59:22,890 --> 00:59:26,400 So I'll go ahead and print out knowledge.formula, 1210 00:59:26,400 --> 00:59:27,525 just to see this in action. 1211 00:59:27,525 --> 00:59:29,608 And I'll go ahead and skip this for now, but we'll 1212 00:59:29,608 --> 00:59:31,240 come back to this in a second. 1213 00:59:31,240 --> 00:59:36,150 Let's print out the knowledge that I know by running Python puzzle.py. 1214 00:59:36,150 --> 00:59:38,070 It's a lot of information, a lot that I have 1215 00:59:38,070 --> 00:59:41,320 to scroll through, because there's 16 different variables all going on. 1216 00:59:41,320 --> 00:59:43,620 But the basic idea if we scroll up to the very top 1217 00:59:43,620 --> 00:59:45,330 is I see my initial information. 1218 00:59:45,330 --> 00:59:48,870 Gildaroy is either in Gryffindor, or Gildaroy is and Hufflepuff, 1219 00:59:48,870 --> 00:59:52,820 or Gildaroy is and Ravenclaw, or Gildaroy is in Slytherin. 1220 00:59:52,820 --> 00:59:55,210 And then way more information as well. 1221 00:59:55,210 --> 00:59:56,280 So this is quite messy. 1222 00:59:56,280 --> 00:59:58,037 More than we really want to be looking at. 1223 00:59:58,037 --> 01:00:00,870 And soon, too, we'll see ways of representing this a little bit more 1224 01:00:00,870 --> 01:00:02,460 nicely using logic. 1225 01:00:02,460 --> 01:00:05,820 But for now, we can just say these are the variables that we're dealing with. 1226 01:00:05,820 --> 01:00:09,830 And now we'd like to add some information. 1227 01:00:09,830 --> 01:00:13,840 So the information we're going to add is Gildaroy is in Gryffindor 1228 01:00:13,840 --> 01:00:14,950 or he is in Ravenclaw. 1229 01:00:14,950 --> 01:00:16,970 So that knowledge was given to us. 1230 01:00:16,970 --> 01:00:20,020 So I'll go ahead and say knowledge.add, and I 1231 01:00:20,020 --> 01:00:30,680 know that either or Gildaroy Gryffindor or Gildaroy Ravenclaw. 1232 01:00:30,680 --> 01:00:33,660 One of those two things must be true. 1233 01:00:33,660 --> 01:00:36,460 I also know that Pomona was not in Slytherin. 1234 01:00:36,460 --> 01:00:39,800 So I can say knowledge.add, not this symbol. 1235 01:00:39,800 --> 01:00:43,010 Not the Pomona Slytherin symbol. 1236 01:00:43,010 --> 01:00:46,970 And then I can add the knowledge that Minerva is in Gryffindor by adding 1237 01:00:46,970 --> 01:00:51,060 the symbol Minerva Gryffindor. 1238 01:00:51,060 --> 01:00:53,490 So those are the pieces of knowledge that I know. 1239 01:00:53,490 --> 01:00:57,210 And this loop here at the bottom just loops over all of my symbols, 1240 01:00:57,210 --> 01:01:01,410 checks to see if the knowledge entails that symbol by calling this model check 1241 01:01:01,410 --> 01:01:02,840 function again. 1242 01:01:02,840 --> 01:01:07,790 And if it does, if we know the symbol is true, we print out the symbol. 1243 01:01:07,790 --> 01:01:11,300 So now I can run Python puzzle.py, and Python 1244 01:01:11,300 --> 01:01:13,190 is going to solve this puzzle for me. 1245 01:01:13,190 --> 01:01:15,800 We're able to conclude that Gildaroy belongs to Ravenclaw, 1246 01:01:15,800 --> 01:01:18,320 Pomona belongs to Hufflepuff, Minerva to Gryffindor, 1247 01:01:18,320 --> 01:01:22,430 and Horace to Slytherin just by encoding this knowledge inside the computer-- 1248 01:01:22,430 --> 01:01:24,680 although, it was quite tedious to do in this case-- 1249 01:01:24,680 --> 01:01:29,280 and as a result, we were able to get the conclusion from that as well. 1250 01:01:29,280 --> 01:01:32,660 And you can imagine this being applied to many sorts of different deductive 1251 01:01:32,660 --> 01:01:33,320 situations. 1252 01:01:33,320 --> 01:01:35,450 So not only these situations where we're trying 1253 01:01:35,450 --> 01:01:38,052 to deal with Harry Potter characters in this puzzle. 1254 01:01:38,052 --> 01:01:40,010 But if you've ever played games like Mastermind 1255 01:01:40,010 --> 01:01:43,340 where you're trying to figure out which order different colors go in 1256 01:01:43,340 --> 01:01:46,700 and trying to make predictions about it, I could tell you, for example. 1257 01:01:46,700 --> 01:01:49,610 Let's play a simplified version of Mastermind where there are four 1258 01:01:49,610 --> 01:01:52,070 colors-- red, blue, green, and yellow-- 1259 01:01:52,070 --> 01:01:55,350 and they're in some order, but I'm not telling you what order. 1260 01:01:55,350 --> 01:01:57,350 You just have to make a guess, and I'll tell you 1261 01:01:57,350 --> 01:01:59,690 of red, blue, green, and yellow, how many of the four 1262 01:01:59,690 --> 01:02:01,520 you got in the right position. 1263 01:02:01,520 --> 01:02:03,620 So a simplified version of this game. 1264 01:02:03,620 --> 01:02:06,210 You might make a guess, like red, blue, green, yellow. 1265 01:02:06,210 --> 01:02:09,650 And I would tell you something like, two of those four 1266 01:02:09,650 --> 01:02:12,447 are in the correct position, but the other two are not. 1267 01:02:12,447 --> 01:02:15,530 Then you could reasonably make a guess and say, all right, let's try this. 1268 01:02:15,530 --> 01:02:17,270 Blue, red, green, yellow. 1269 01:02:17,270 --> 01:02:19,100 Try switching two of them around. 1270 01:02:19,100 --> 01:02:22,190 And this time maybe I tell you, you know what, none of those 1271 01:02:22,190 --> 01:02:23,780 are in the correct position. 1272 01:02:23,780 --> 01:02:26,090 And the question then is, all right, what is 1273 01:02:26,090 --> 01:02:28,280 the correct order of these four colors? 1274 01:02:28,280 --> 01:02:30,530 And we, as humans, could begin to reason this through. 1275 01:02:30,530 --> 01:02:34,820 All right, well, if none of these were correct, but two of these were correct, 1276 01:02:34,820 --> 01:02:37,730 well, it must have been because I switched the red and the blue. 1277 01:02:37,730 --> 01:02:39,990 Which means red and blue here must be correct. 1278 01:02:39,990 --> 01:02:42,240 Which means green and yellow are probably not correct. 1279 01:02:42,240 --> 01:02:44,760 You can begin to do this sort of deductive reasoning. 1280 01:02:44,760 --> 01:02:48,230 We can also equivalently try and take this and encode it inside 1281 01:02:48,230 --> 01:02:49,700 of our computer as well. 1282 01:02:49,700 --> 01:02:52,310 And it's going to be very similar to the logic puzzle 1283 01:02:52,310 --> 01:02:53,678 that we just did a moment ago. 1284 01:02:53,678 --> 01:02:56,720 So I won't spend too much time on this code because it is fairly similar. 1285 01:02:56,720 --> 01:02:59,210 But again, we have a whole bunch of colors, 1286 01:02:59,210 --> 01:03:02,880 and four different positions in which those colors can be. 1287 01:03:02,880 --> 01:03:04,730 And then we have some additional knowledge. 1288 01:03:04,730 --> 01:03:06,410 And I encode all of that knowledge. 1289 01:03:06,410 --> 01:03:09,360 And you can take a look at this code on your own time. 1290 01:03:09,360 --> 01:03:12,080 But I just want to demonstrate that when we run this code, 1291 01:03:12,080 --> 01:03:17,100 run Python mastermind.py and run and see what we get, 1292 01:03:17,100 --> 01:03:21,170 we ultimately are able to compute red in the zero position, 1293 01:03:21,170 --> 01:03:23,750 blue in the one position, yellow in the two position, 1294 01:03:23,750 --> 01:03:28,307 and green in the three position as the ordering of those symbols. 1295 01:03:28,307 --> 01:03:30,140 Now, ultimately, what you might have noticed 1296 01:03:30,140 --> 01:03:32,870 is this process was taking quite a long time. 1297 01:03:32,870 --> 01:03:36,650 And, in fact, model checking is not a particularly efficient algorithm. 1298 01:03:36,650 --> 01:03:38,630 What I need to do in order to model check 1299 01:03:38,630 --> 01:03:41,070 is take all of my possible different variables 1300 01:03:41,070 --> 01:03:43,790 and enumerate all of the possibilities that they could be in. 1301 01:03:43,790 --> 01:03:48,320 If I have n variables, I have 2 to the n possible worlds 1302 01:03:48,320 --> 01:03:51,740 that I need to be looking through in order to perform this model checking 1303 01:03:51,740 --> 01:03:52,370 algorithm. 1304 01:03:52,370 --> 01:03:54,740 And this is probably not tractable, especially 1305 01:03:54,740 --> 01:03:57,770 as we start to get to much larger and larger sets of data 1306 01:03:57,770 --> 01:04:00,620 where you have many, many more variables that are at play. 1307 01:04:00,620 --> 01:04:03,440 Right here we only have a relatively small number of variables, 1308 01:04:03,440 --> 01:04:05,840 so this sort of approach can actually work. 1309 01:04:05,840 --> 01:04:08,120 But as the number of variables increases, 1310 01:04:08,120 --> 01:04:11,540 model checking becomes less and less good of a way of trying 1311 01:04:11,540 --> 01:04:14,050 to solve these sorts of problems. 1312 01:04:14,050 --> 01:04:16,550 So while it might have been OK for something like Mastermind 1313 01:04:16,550 --> 01:04:18,842 to conclude that this is, indeed, the correct sequence, 1314 01:04:18,842 --> 01:04:22,010 where all four are in the correct position, what we'd like to do 1315 01:04:22,010 --> 01:04:25,910 is come up with some better ways to be able to make inferences, rather 1316 01:04:25,910 --> 01:04:28,910 than just enumerate all of the possibilities. 1317 01:04:28,910 --> 01:04:33,290 And to do so, what we'll transition to next is the idea of inference rules. 1318 01:04:33,290 --> 01:04:37,490 Some sort of rules that we can apply to take knowledge that already exists 1319 01:04:37,490 --> 01:04:40,310 and translate it into new forms of knowledge. 1320 01:04:40,310 --> 01:04:42,710 And the general way we'll structure inference rule 1321 01:04:42,710 --> 01:04:45,140 is by having a horizontal line here. 1322 01:04:45,140 --> 01:04:48,560 Anything above the line is going to represent a premise, something 1323 01:04:48,560 --> 01:04:50,260 that we know to be true. 1324 01:04:50,260 --> 01:04:52,970 And then anything below the line will be the conclusion 1325 01:04:52,970 --> 01:04:56,552 that we can arrive at after we apply the logic, 1326 01:04:56,552 --> 01:04:59,010 or from the inference rule that we're going to demonstrate. 1327 01:04:59,010 --> 01:05:00,440 So we'll do some of these inference rules 1328 01:05:00,440 --> 01:05:03,320 by demonstrating them in English first, but then translating them 1329 01:05:03,320 --> 01:05:05,420 into the world of propositional logic so you 1330 01:05:05,420 --> 01:05:09,120 can see what those inference rules actually look like. 1331 01:05:09,120 --> 01:05:11,300 So for example, let's imagine that I have access 1332 01:05:11,300 --> 01:05:13,040 to two pieces of information. 1333 01:05:13,040 --> 01:05:15,720 I know, for example, that if it is raining, 1334 01:05:15,720 --> 01:05:18,340 then Harry is inside, for example. 1335 01:05:18,340 --> 01:05:21,167 And let's say I also know it is raining. 1336 01:05:21,167 --> 01:05:23,750 Then most of us could reasonably then look at this information 1337 01:05:23,750 --> 01:05:28,180 and conclude that, all right, Harry must be inside. 1338 01:05:28,180 --> 01:05:31,450 This inference rule is known as modus ponens, 1339 01:05:31,450 --> 01:05:34,210 and it's phrased more formally in logic as this. 1340 01:05:34,210 --> 01:05:38,620 If we know that alpha implies beta, in other words, if alpha, 1341 01:05:38,620 --> 01:05:42,410 then beta, and we also know that alpha is true, 1342 01:05:42,410 --> 01:05:45,950 then we should be able to conclude that beta is also true. 1343 01:05:45,950 --> 01:05:49,960 We can apply this inference rule to take these two pieces of information 1344 01:05:49,960 --> 01:05:52,250 and generate this new piece of information. 1345 01:05:52,250 --> 01:05:55,390 Notice that this is a totally different approach from the model checking 1346 01:05:55,390 --> 01:05:58,810 approach, where the approach was look at all of the possible worlds 1347 01:05:58,810 --> 01:06:01,000 and see what's true in each of these worlds. 1348 01:06:01,000 --> 01:06:03,520 Here, we're not dealing with any specific world. 1349 01:06:03,520 --> 01:06:05,830 We're just dealing with the knowledge that we know 1350 01:06:05,830 --> 01:06:08,770 and what conclusions we can arrive at based on that knowledge. 1351 01:06:08,770 --> 01:06:14,320 That I know that A implies B, and I know A, and the conclusion is B. 1352 01:06:14,320 --> 01:06:16,950 And this should seem like a relatively obvious rule. 1353 01:06:16,950 --> 01:06:20,410 But of course, if alpha than beta, and we know alpha, 1354 01:06:20,410 --> 01:06:23,570 then we should be able to conclude that beta is also true. 1355 01:06:23,570 --> 01:06:26,867 And that's going to be true for many, maybe even all of the inference rules 1356 01:06:26,867 --> 01:06:27,950 that we'll take a look at. 1357 01:06:27,950 --> 01:06:29,992 You should be able to look at them and say, yeah, 1358 01:06:29,992 --> 01:06:31,448 of course that's going to be true. 1359 01:06:31,448 --> 01:06:33,490 But it's putting these all together, figuring out 1360 01:06:33,490 --> 01:06:35,650 the right combination of inference rules that 1361 01:06:35,650 --> 01:06:39,970 can be applied that ultimately is going to allow us to generate 1362 01:06:39,970 --> 01:06:42,730 interesting knowledge inside of our AI. 1363 01:06:42,730 --> 01:06:45,700 So that's modus ponens, this application of implication. 1364 01:06:45,700 --> 01:06:48,910 That if we know alpha, and we know that alpha implies beta, 1365 01:06:48,910 --> 01:06:51,508 then we can conclude beta. 1366 01:06:51,508 --> 01:06:53,050 Let's take a look at another example. 1367 01:06:53,050 --> 01:06:54,008 Fairly straightforward. 1368 01:06:54,008 --> 01:06:56,860 Something like Harry is friends with Ron and Hermione. 1369 01:06:56,860 --> 01:06:59,110 Based on that information, we can reasonably 1370 01:06:59,110 --> 01:07:01,210 conclude Harry is friends with Hermione. 1371 01:07:01,210 --> 01:07:03,070 That must also be true. 1372 01:07:03,070 --> 01:07:06,190 And this inference rule is known as and elimination. 1373 01:07:06,190 --> 01:07:10,060 And what and elimination says, is that if we have a situation where 1374 01:07:10,060 --> 01:07:13,240 alpha and beta are both true-- 1375 01:07:13,240 --> 01:07:15,840 I have information alpha and beta-- 1376 01:07:15,840 --> 01:07:18,730 well, then just alpha is true. 1377 01:07:18,730 --> 01:07:20,850 Or likewise, just beta is true. 1378 01:07:20,850 --> 01:07:24,100 That if I know that both parts are true, then one of those parts 1379 01:07:24,100 --> 01:07:25,330 must also be true. 1380 01:07:25,330 --> 01:07:28,660 Again, something obvious from the point of view of human intuition. 1381 01:07:28,660 --> 01:07:31,480 But a computer needs to be told this kind of information. 1382 01:07:31,480 --> 01:07:33,280 To be able to apply the inference rule, we 1383 01:07:33,280 --> 01:07:35,988 need to tell the computer that this is an inference rule that you 1384 01:07:35,988 --> 01:07:38,320 can apply so the computer has access to it, 1385 01:07:38,320 --> 01:07:41,350 and is able to use it in order to translate information 1386 01:07:41,350 --> 01:07:44,165 from one form to another. 1387 01:07:44,165 --> 01:07:46,540 In addition to that, let's take a look at another example 1388 01:07:46,540 --> 01:07:47,800 of an inference rule. 1389 01:07:47,800 --> 01:07:53,055 Something like, it is not true that Harry did not pass the test. 1390 01:07:53,055 --> 01:07:55,180 Bit of a tricky sentence to parse so read it again. 1391 01:07:55,180 --> 01:07:59,320 It is not true, or it is false, that Harry did not pass the test. 1392 01:07:59,320 --> 01:08:03,130 Well, if it is false that Harry did not pass the test, 1393 01:08:03,130 --> 01:08:07,270 then the only reasonable conclusion is that Harry did pass the test. 1394 01:08:07,270 --> 01:08:09,460 And so this, instead of being and elimination, 1395 01:08:09,460 --> 01:08:11,920 is what we call double negation elimination. 1396 01:08:11,920 --> 01:08:14,860 That if we have two negatives inside of our premise, then 1397 01:08:14,860 --> 01:08:16,345 we can just remove them altogether. 1398 01:08:16,345 --> 01:08:17,470 They cancel each other out. 1399 01:08:17,470 --> 01:08:21,760 One turns true to false, and the other one turns false back into true. 1400 01:08:21,760 --> 01:08:23,649 Phrased a little bit more formally, we say 1401 01:08:23,649 --> 01:08:29,050 that if the premise is not not alpha, then the conclusion we can draw 1402 01:08:29,050 --> 01:08:30,130 is just alpha. 1403 01:08:30,130 --> 01:08:32,767 We can say that alpha is true. 1404 01:08:32,767 --> 01:08:34,600 We'll take a look at a couple more of these. 1405 01:08:34,600 --> 01:08:40,240 If I have it is raining, then Harry is inside, how do I reframe this? 1406 01:08:40,240 --> 01:08:42,100 Well, this one is a little bit trickier. 1407 01:08:42,100 --> 01:08:45,399 But if I know if it is raining, then Harry is inside, 1408 01:08:45,399 --> 01:08:48,160 then I conclude one of two things must be true. 1409 01:08:48,160 --> 01:08:52,404 Either it is not raining, or Harry is inside. 1410 01:08:52,404 --> 01:08:55,029 Now, this one's trickier, so let's think about it a little bit. 1411 01:08:55,029 --> 01:08:58,710 This first premise here, if it is raining, then Harry is inside, 1412 01:08:58,710 --> 01:09:03,520 is saying that if I know that it is raining, then Harry must be inside. 1413 01:09:03,520 --> 01:09:06,160 So what is the other possible case? 1414 01:09:06,160 --> 01:09:11,109 Well, if Harry is not inside, then I know that it must not be raining. 1415 01:09:11,109 --> 01:09:13,880 So one of those two situations must be true. 1416 01:09:13,880 --> 01:09:19,220 Either it's not raining, or it is raining, in which case Harry is inside. 1417 01:09:19,220 --> 01:09:24,100 So the conclusion I can draw is either it is not raining, or it is raining, 1418 01:09:24,100 --> 01:09:27,350 so therefore, Harry is inside. 1419 01:09:27,350 --> 01:09:32,350 So this is a way to translate if/then statements into or statements. 1420 01:09:32,350 --> 01:09:35,220 And this is known as implication elimination. 1421 01:09:35,220 --> 01:09:37,720 And this is similar to what we actually did in the beginning 1422 01:09:37,720 --> 01:09:41,560 when we were first looking at those very first sentences about Harry and Hagrid 1423 01:09:41,560 --> 01:09:42,307 and Dumbledore. 1424 01:09:42,307 --> 01:09:44,140 And phrased a little bit more formally, this 1425 01:09:44,140 --> 01:09:46,479 says that if I have the implication alpha 1426 01:09:46,479 --> 01:09:51,970 implies beta, that I can draw the conclusion that either not alpha 1427 01:09:51,970 --> 01:09:53,470 or beta. 1428 01:09:53,470 --> 01:09:55,190 Because there are only two possibilities. 1429 01:09:55,190 --> 01:09:58,370 Either alpha is true or alpha is not true. 1430 01:09:58,370 --> 01:10:01,540 So one of those possibilities is alpha is not true. 1431 01:10:01,540 --> 01:10:04,660 But if alpha is true, well, then we can draw the conclusion 1432 01:10:04,660 --> 01:10:05,990 that beta must be true. 1433 01:10:05,990 --> 01:10:12,350 So either alpha is not true, or alpha is true, in which case beta is also true. 1434 01:10:12,350 --> 01:10:16,060 So this is one way to turn an implication into just a statement about 1435 01:10:16,060 --> 01:10:16,780 or. 1436 01:10:16,780 --> 01:10:19,450 In addition to eliminating implications, we can also 1437 01:10:19,450 --> 01:10:21,980 eliminate biconditionals as well. 1438 01:10:21,980 --> 01:10:23,500 So let's take an English example. 1439 01:10:23,500 --> 01:10:27,930 Something like, it is raining if and only if Harry is inside. 1440 01:10:27,930 --> 01:10:31,280 And this if and only if really sounds like that biconditional, 1441 01:10:31,280 --> 01:10:35,520 that double arrow sign that we saw in propositional logic not too long ago. 1442 01:10:35,520 --> 01:10:38,160 And what does this actually mean if we were to translate this? 1443 01:10:38,160 --> 01:10:42,090 Well, this means that if it is raining, then Harry is inside. 1444 01:10:42,090 --> 01:10:44,590 And if Harry is inside, then it is raining. 1445 01:10:44,590 --> 01:10:47,440 The implication goes both ways. 1446 01:10:47,440 --> 01:10:50,280 And this is what we would call biconditional elimination. 1447 01:10:50,280 --> 01:10:54,237 That I can take a biconditional, A if and only if B, 1448 01:10:54,237 --> 01:10:56,070 and translate that into something like this. 1449 01:10:56,070 --> 01:11:02,130 A implies B, and B implies A. So many of these inference rules 1450 01:11:02,130 --> 01:11:04,710 are taking logic that uses certain symbols 1451 01:11:04,710 --> 01:11:08,280 and turning them into different symbols, taking an implication 1452 01:11:08,280 --> 01:11:09,370 and turning it into an or. 1453 01:11:09,370 --> 01:11:12,960 Or taking a biconditional and turning it into implication. 1454 01:11:12,960 --> 01:11:15,930 And another example of it would be something like this. 1455 01:11:15,930 --> 01:11:20,420 It is not true that both Harry and Ron passed the test. 1456 01:11:20,420 --> 01:11:22,170 Well, all right, how do we translate that? 1457 01:11:22,170 --> 01:11:23,230 What does that mean? 1458 01:11:23,230 --> 01:11:27,120 Well, if it is not true that both of them passed the test, 1459 01:11:27,120 --> 01:11:31,200 well, then the reasonable conclusion we might draw is that at least one of them 1460 01:11:31,200 --> 01:11:32,370 didn't pass the test. 1461 01:11:32,370 --> 01:11:35,580 So the conclusion is either Harry did not pass the test, 1462 01:11:35,580 --> 01:11:38,050 or Ron did not pass the test, or both. 1463 01:11:38,050 --> 01:11:39,540 This is not an exclusive or. 1464 01:11:39,540 --> 01:11:43,980 But if it is true that it is not true, that both Harry and Ron passed 1465 01:11:43,980 --> 01:11:47,160 the test, well, then either Harry didn't pass the test 1466 01:11:47,160 --> 01:11:49,580 or Ron didn't pass the test. 1467 01:11:49,580 --> 01:11:52,320 And this type of law is one of De Morgan's laws. 1468 01:11:52,320 --> 01:11:57,330 Quite famous in logic, where the idea is that we can turn an and into an or. 1469 01:11:57,330 --> 01:12:00,690 We can take this and, that both Harry and Ron passed the test, 1470 01:12:00,690 --> 01:12:04,140 and turn it into an or by moving the nots around. 1471 01:12:04,140 --> 01:12:07,680 So if it is not true that Harry and Ron passed the test, 1472 01:12:07,680 --> 01:12:10,110 well, then either Harry did not pass the test, 1473 01:12:10,110 --> 01:12:13,200 or Ron did not pass the test either. 1474 01:12:13,200 --> 01:12:16,590 And the way we frame that more formally using logic is to say this. 1475 01:12:16,590 --> 01:12:21,690 If it is not true that alpha and beta, well, then either not 1476 01:12:21,690 --> 01:12:24,598 alpha or not beta. 1477 01:12:24,598 --> 01:12:26,640 The way I like to think about this is that if you 1478 01:12:26,640 --> 01:12:29,550 have a negation in front of an and expression, 1479 01:12:29,550 --> 01:12:32,190 you move the negation inwards, so to speak. 1480 01:12:32,190 --> 01:12:36,210 Moving the negation into each of these individual sentences, 1481 01:12:36,210 --> 01:12:38,920 and then flip the and into an or. 1482 01:12:38,920 --> 01:12:42,090 So the negation moves inwards, and the and flips into an or. 1483 01:12:42,090 --> 01:12:47,908 So I go from not A and B, to not A or not B. 1484 01:12:47,908 --> 01:12:49,950 And there's actually a reverse of De Morgan's law 1485 01:12:49,950 --> 01:12:52,620 that goes in the other direction for something like this. 1486 01:12:52,620 --> 01:12:56,550 If I say, it is not true that Harry or Ron passed the test, 1487 01:12:56,550 --> 01:13:00,450 meaning neither of them passed the test, well, then the conclusion I can draw 1488 01:13:00,450 --> 01:13:05,440 is that Harry did not pass the test, and Ron did not pass the test. 1489 01:13:05,440 --> 01:13:08,940 So in this case, instead of turning an and into an or, we're turning an 1490 01:13:08,940 --> 01:13:10,830 or into an and. 1491 01:13:10,830 --> 01:13:12,030 But the idea is the same. 1492 01:13:12,030 --> 01:13:15,140 And this, again, is another example of De Morgan's laws. 1493 01:13:15,140 --> 01:13:20,007 And the way that works is that if I have not A or B this time, 1494 01:13:20,007 --> 01:13:21,340 the same logic's going to apply. 1495 01:13:21,340 --> 01:13:23,700 I'm going to move the negation inwards, and I'm 1496 01:13:23,700 --> 01:13:26,880 going to flip, this time, flip the or into an and. 1497 01:13:26,880 --> 01:13:32,730 So if not A or B, meaning it is not true that A or B, or alpha or beta, 1498 01:13:32,730 --> 01:13:36,990 then I can say not alpha and not beta. 1499 01:13:36,990 --> 01:13:40,630 Moving the negation inwards in order to make that conclusion. 1500 01:13:40,630 --> 01:13:41,880 So those are De Morgan's laws. 1501 01:13:41,880 --> 01:13:45,005 And a couple of other inference rules that are worth just taking a look at, 1502 01:13:45,005 --> 01:13:47,680 one is the distributive law that works this way. 1503 01:13:47,680 --> 01:13:53,910 So if I have alpha and beta or gamma, well, then much in the same way 1504 01:13:53,910 --> 01:13:56,370 that you can use, in math, use distributive laws 1505 01:13:56,370 --> 01:13:59,750 to distribute operands, like addition and multiplication, 1506 01:13:59,750 --> 01:14:01,350 I can do a similar thing here. 1507 01:14:01,350 --> 01:14:05,400 Where I can say if alpha and beta or gamma, 1508 01:14:05,400 --> 01:14:10,890 then I can say something like, alpha and beta, or alpha and gamma. 1509 01:14:10,890 --> 01:14:15,407 That I've been able to distribute this and sign throughout this expression. 1510 01:14:15,407 --> 01:14:17,490 So this is an example of the distributive property 1511 01:14:17,490 --> 01:14:21,240 or the distributive law, as applied to logic in much the same way 1512 01:14:21,240 --> 01:14:24,120 that you would distribute like a multiplication over the addition 1513 01:14:24,120 --> 01:14:26,560 of something, for example. 1514 01:14:26,560 --> 01:14:28,060 This works the other way, too. 1515 01:14:28,060 --> 01:14:32,250 So if, for example, I have alpha or beta and gamma, 1516 01:14:32,250 --> 01:14:34,590 I can distribute the or throughout the expression. 1517 01:14:34,590 --> 01:14:38,680 I can say alpha or beta, and alpha or gamma. 1518 01:14:38,680 --> 01:14:40,800 So the distributive law works in that way, too. 1519 01:14:40,800 --> 01:14:44,640 And it's helpful if I want to take an or and move it into the expression. 1520 01:14:44,640 --> 01:14:47,490 And we'll see an example soon of why it is that we might actually 1521 01:14:47,490 --> 01:14:50,440 care to do something like that. 1522 01:14:50,440 --> 01:14:50,940 All right. 1523 01:14:50,940 --> 01:14:53,920 So now we've seen a lot of different inference rules. 1524 01:14:53,920 --> 01:14:57,660 And the question now is how can we use those inference rules to actually 1525 01:14:57,660 --> 01:14:59,040 try and draw some conclusions? 1526 01:14:59,040 --> 01:15:01,440 To actually try and prove something about entailment, 1527 01:15:01,440 --> 01:15:03,720 proving that given some initial knowledge base, 1528 01:15:03,720 --> 01:15:08,890 we would like to find some way to prove that a query is true. 1529 01:15:08,890 --> 01:15:10,830 Well, one way to think about it is actually 1530 01:15:10,830 --> 01:15:12,900 to think back to what we talked about last time, 1531 01:15:12,900 --> 01:15:14,790 when we talked about search problems. 1532 01:15:14,790 --> 01:15:17,700 Recall again that search problems have some sort of initial state. 1533 01:15:17,700 --> 01:15:20,520 They have actions that you can take from one state to another, 1534 01:15:20,520 --> 01:15:23,610 as defined by a transition model that tells you how to get from one state 1535 01:15:23,610 --> 01:15:24,640 to another. 1536 01:15:24,640 --> 01:15:27,120 We talked about testing to see if you had a goal. 1537 01:15:27,120 --> 01:15:31,530 And then some path cost function to see how many steps did you have to take, 1538 01:15:31,530 --> 01:15:35,340 or how costly was the solution that you found. 1539 01:15:35,340 --> 01:15:37,380 Now that we have these inference rules that 1540 01:15:37,380 --> 01:15:41,040 take some set of sentences and propositional logic 1541 01:15:41,040 --> 01:15:44,700 and get us some new set of sentences in propositional logic, 1542 01:15:44,700 --> 01:15:49,050 we can actually treat those sentences, or those sets of sentences, 1543 01:15:49,050 --> 01:15:51,610 as states inside of a search problem. 1544 01:15:51,610 --> 01:15:54,060 So if we want to prove that some query is true, 1545 01:15:54,060 --> 01:15:56,610 prove that some logical theorem is true, we 1546 01:15:56,610 --> 01:16:00,150 can treat theorem proving as a form of a search problem. 1547 01:16:00,150 --> 01:16:03,420 I can say that we begin in some initial state, 1548 01:16:03,420 --> 01:16:06,330 where that initial state is the knowledge base that I begin with. 1549 01:16:06,330 --> 01:16:09,900 The set of all of the sentences that I know to be true. 1550 01:16:09,900 --> 01:16:11,580 What actions are available to me? 1551 01:16:11,580 --> 01:16:13,830 Well, the actions are any of the inference rules 1552 01:16:13,830 --> 01:16:16,380 that I can apply at any given time. 1553 01:16:16,380 --> 01:16:20,730 The transition model just tells me after I apply the inference rule, 1554 01:16:20,730 --> 01:16:23,290 here is the new set of all of the knowledge that I have, 1555 01:16:23,290 --> 01:16:26,568 which will be the old set of knowledge, plus some additional inference 1556 01:16:26,568 --> 01:16:28,860 that I've been able to draw, much as in the same way we 1557 01:16:28,860 --> 01:16:31,490 saw what we got when we applied those inference rules 1558 01:16:31,490 --> 01:16:33,000 and got some sort of conclusion. 1559 01:16:33,000 --> 01:16:35,910 That conclusion gets added to our knowledge base, 1560 01:16:35,910 --> 01:16:38,550 and our transition model will encode that. 1561 01:16:38,550 --> 01:16:39,500 What is the goal test? 1562 01:16:39,500 --> 01:16:41,250 Well, our goal test is, you know, checking 1563 01:16:41,250 --> 01:16:44,790 to see if we have proved the statement we're trying to prove. 1564 01:16:44,790 --> 01:16:49,170 If the thing we're trying to prove is inside of our knowledge base. 1565 01:16:49,170 --> 01:16:52,230 And the path cost function, the thing we're trying to minimize, 1566 01:16:52,230 --> 01:16:55,260 is maybe the number of inference rules that we needed to use. 1567 01:16:55,260 --> 01:16:59,160 The number of steps, so to speak, inside of our proof. 1568 01:16:59,160 --> 01:17:02,070 And so here we've been able to apply the same types of ideas 1569 01:17:02,070 --> 01:17:04,140 that we saw last time with search problems, 1570 01:17:04,140 --> 01:17:06,840 to something like trying to prove something about knowledge 1571 01:17:06,840 --> 01:17:11,048 by taking our knowledge and framing it in terms that we can understand 1572 01:17:11,048 --> 01:17:12,840 as a search problem, with an initial state, 1573 01:17:12,840 --> 01:17:15,210 with actions, with a transition model. 1574 01:17:15,210 --> 01:17:17,080 So this shows a couple of things. 1575 01:17:17,080 --> 01:17:19,310 One being how versatile search problems are. 1576 01:17:19,310 --> 01:17:23,130 That they can be the same types of algorithms that we use to solve a maze, 1577 01:17:23,130 --> 01:17:26,770 or figure out how to get from point A to point B. Inside of driving directions, 1578 01:17:26,770 --> 01:17:30,390 for example, can also be used as a theorem proofing method. 1579 01:17:30,390 --> 01:17:32,610 Of taking some sort of starting knowledge base 1580 01:17:32,610 --> 01:17:36,210 and trying to prove something about that knowledge. 1581 01:17:36,210 --> 01:17:39,420 So this, yet again, is a second way, in addition to model checking, 1582 01:17:39,420 --> 01:17:43,000 to try and prove that certain statements are true. 1583 01:17:43,000 --> 01:17:46,425 But it turns out there's yet another way that we can try and apply inference, 1584 01:17:46,425 --> 01:17:48,800 and we'll talk about this now, which is not the only way, 1585 01:17:48,800 --> 01:17:50,700 but certainly one of the most common. 1586 01:17:50,700 --> 01:17:52,850 Which is known as resolution. 1587 01:17:52,850 --> 01:17:55,890 And resolution is based on another inference rule 1588 01:17:55,890 --> 01:17:57,140 that we'll take a look at now. 1589 01:17:57,140 --> 01:17:59,090 Quite a powerful inference rule that will 1590 01:17:59,090 --> 01:18:03,110 let us prove anything that can be proven about a knowledge base. 1591 01:18:03,110 --> 01:18:05,730 And it's based on this basic idea. 1592 01:18:05,730 --> 01:18:09,680 Let's say I know that either Ron is in the Great Hall, 1593 01:18:09,680 --> 01:18:12,240 or Hermione is in the library. 1594 01:18:12,240 --> 01:18:16,790 And let's say I also know that Ron is not in the Great Hall. 1595 01:18:16,790 --> 01:18:20,480 Based on those two pieces of information, what can I conclude? 1596 01:18:20,480 --> 01:18:24,470 Well, I could pretty reasonably conclude that Hermione must be in the library. 1597 01:18:24,470 --> 01:18:25,560 How do I know that? 1598 01:18:25,560 --> 01:18:29,360 Well, it's because these two statements, these two, what we'll call, 1599 01:18:29,360 --> 01:18:32,930 complementary literals-- literals that complement each other, 1600 01:18:32,930 --> 01:18:34,970 they are opposites of each other-- 1601 01:18:34,970 --> 01:18:36,920 seem to conflict with each other. 1602 01:18:36,920 --> 01:18:39,800 This sentence tells us that either Ron is in the Great Hall 1603 01:18:39,800 --> 01:18:41,970 or Hermione is in the library. 1604 01:18:41,970 --> 01:18:44,420 So if we know that Ron is not in the Great Hall, 1605 01:18:44,420 --> 01:18:49,880 that conflicts with this one, which means Hermione must be in the library. 1606 01:18:49,880 --> 01:18:54,150 And this we can frame as a more general rule, known as the unit resolution 1607 01:18:54,150 --> 01:18:54,650 rule. 1608 01:18:54,650 --> 01:19:01,880 A rule that says that if we have P or Q, and we also know not P, well, then 1609 01:19:01,880 --> 01:19:04,700 from that we can reasonably conclude Q. 1610 01:19:04,700 --> 01:19:08,280 That if P or Q are true, and we know that P is not true, 1611 01:19:08,280 --> 01:19:12,170 the only possibility is for Q to then be true. 1612 01:19:12,170 --> 01:19:14,660 And this, it turns out, is quite a powerful inference rule 1613 01:19:14,660 --> 01:19:17,450 in terms of what it can do, in part because we can quickly 1614 01:19:17,450 --> 01:19:19,260 start to generalize this rule. 1615 01:19:19,260 --> 01:19:23,360 This Q right here doesn't need to just be a single propositional symbol. 1616 01:19:23,360 --> 01:19:26,720 It could be multiple, all chained together in a single clause, 1617 01:19:26,720 --> 01:19:27,690 as we'll call it. 1618 01:19:27,690 --> 01:19:32,750 So if I had something like P or Q1, or Q2, or Q3, so on and so forth, 1619 01:19:32,750 --> 01:19:36,740 up until Qn, so I had n, different, other variables, 1620 01:19:36,740 --> 01:19:40,910 and I have not P, well, then what happens when these two complement 1621 01:19:40,910 --> 01:19:44,990 each other is that these two clauses resolve, so to speak, 1622 01:19:44,990 --> 01:19:50,570 to produce a new clause that is just Q1 or Q2, all the way up to Qn. 1623 01:19:50,570 --> 01:19:53,890 And in an or, the order of the arguments in the or doesn't actually matter. 1624 01:19:53,890 --> 01:19:55,240 The P doesn't need to be the first thing. 1625 01:19:55,240 --> 01:19:56,532 It could've been in the middle. 1626 01:19:56,532 --> 01:20:00,020 But the idea here is that if I have P in one clause, 1627 01:20:00,020 --> 01:20:02,060 and not P and the other clause, well, then 1628 01:20:02,060 --> 01:20:05,090 I know that one of these remaining things must be true. 1629 01:20:05,090 --> 01:20:08,900 I've resolved them in order to produce a new clause. 1630 01:20:08,900 --> 01:20:12,870 But it turns out we can generalize this idea even further, in fact, 1631 01:20:12,870 --> 01:20:16,950 and display even more power that we can have with this resolution rule. 1632 01:20:16,950 --> 01:20:18,768 So let's take another example. 1633 01:20:18,768 --> 01:20:21,560 Let's say, for instance, that I know the same piece of information, 1634 01:20:21,560 --> 01:20:25,730 that either Ron is in the Great Hall or Hermione is in the library. 1635 01:20:25,730 --> 01:20:27,980 And the second piece of information I know 1636 01:20:27,980 --> 01:20:33,780 is that Ron is not in the Great Hall or Harry is sleeping. 1637 01:20:33,780 --> 01:20:35,840 So it's not just a single piece of information. 1638 01:20:35,840 --> 01:20:40,070 I have two different clauses, and we'll define clauses more precisely 1639 01:20:40,070 --> 01:20:41,760 in just a moment. 1640 01:20:41,760 --> 01:20:42,890 What do I know here? 1641 01:20:42,890 --> 01:20:46,670 Well, again, for any propositional symbol, like Ron is in the Great Hall, 1642 01:20:46,670 --> 01:20:48,720 there are only two possibilities. 1643 01:20:48,720 --> 01:20:52,820 Either Ron is in the Great Hall, in which case, based on resolution, 1644 01:20:52,820 --> 01:20:55,220 we know that Harry must be sleeping. 1645 01:20:55,220 --> 01:20:58,670 Or Ron is not in the Great Hall, in which case 1646 01:20:58,670 --> 01:21:03,590 we know, based on the same rule, that Hermione must be in the library. 1647 01:21:03,590 --> 01:21:05,600 Based on those two things in combination, 1648 01:21:05,600 --> 01:21:08,240 I can say, based on these two premises, that I 1649 01:21:08,240 --> 01:21:14,660 can conclude that either Hermione is in the library or Harry is sleeping. 1650 01:21:14,660 --> 01:21:17,580 So again, because these two conflict with each other, 1651 01:21:17,580 --> 01:21:19,975 I know that one of these two must be true. 1652 01:21:19,975 --> 01:21:22,850 And you can take a closer look and try and reason through that logic. 1653 01:21:22,850 --> 01:21:26,690 Make sure you convince yourself that you believe this conclusion. 1654 01:21:26,690 --> 01:21:29,600 Stated more generally, we can name this resolution rule 1655 01:21:29,600 --> 01:21:32,990 by saying that if we know P or Q is true, 1656 01:21:32,990 --> 01:21:37,370 and we also know that not P or R is true, 1657 01:21:37,370 --> 01:21:41,480 we resolve these two clauses together to get a new clause, Q 1658 01:21:41,480 --> 01:21:45,630 or R. That either Q or R must be true. 1659 01:21:45,630 --> 01:21:48,200 And again, much as in the last case, Q and R 1660 01:21:48,200 --> 01:21:51,050 don't need to just be single propositional symbols. 1661 01:21:51,050 --> 01:21:52,460 It could be multiple symbols. 1662 01:21:52,460 --> 01:21:57,020 So if I had a rule that had a P or Q1 or Q2 or Q3, so on and so forth, 1663 01:21:57,020 --> 01:21:59,990 up until Qn, where n is just some number. 1664 01:21:59,990 --> 01:22:06,820 And likewise, I had not P or R1 or F2, so on and so forth, up until Rm, m, 1665 01:22:06,820 --> 01:22:09,620 where m, again, is just some other number, 1666 01:22:09,620 --> 01:22:13,970 I can resolve these two clauses together to get one of these must be true. 1667 01:22:13,970 --> 01:22:15,740 Q1 or Q2, up until Qn. 1668 01:22:15,740 --> 01:22:18,980 Or R1 or R2, up until Rm. 1669 01:22:18,980 --> 01:22:23,810 And this is just a generalization of that same rule we saw before. 1670 01:22:23,810 --> 01:22:27,440 Each of these things here we're going to call a clause. 1671 01:22:27,440 --> 01:22:32,020 Where a clause is formally defined as a disjunction of literals. 1672 01:22:32,020 --> 01:22:35,990 Where a disjunction means it's a bunch of things that are connected with or. 1673 01:22:35,990 --> 01:22:38,390 Disjunction means things connected with or. 1674 01:22:38,390 --> 01:22:41,410 Conjunction, meanwhile, is things connected with and. 1675 01:22:41,410 --> 01:22:45,410 And a literal is either a propositional symbol or the opposite 1676 01:22:45,410 --> 01:22:46,680 of a propositional symbol. 1677 01:22:46,680 --> 01:22:50,420 So it's something like P or Q, or not P or not Q, 1678 01:22:50,420 --> 01:22:54,620 those are all propositional symbols, or not of the propositional symbols, 1679 01:22:54,620 --> 01:22:57,003 and we call those literals. 1680 01:22:57,003 --> 01:22:58,670 So a clause is just something like this. 1681 01:22:58,670 --> 01:23:02,240 P or Q or R, for example. 1682 01:23:02,240 --> 01:23:04,730 Meanwhile, what this gives us an ability to do 1683 01:23:04,730 --> 01:23:08,810 is it gives us an ability to turn logic, any logical sentence, 1684 01:23:08,810 --> 01:23:12,260 into something called conjunctive normal form. 1685 01:23:12,260 --> 01:23:15,700 A conjunctive normal form sentence is a logical sentence 1686 01:23:15,700 --> 01:23:18,530 that is a conjunction of clauses. 1687 01:23:18,530 --> 01:23:23,090 Recall again, conjunction means things are connected to one another using and. 1688 01:23:23,090 --> 01:23:25,580 And so a conjunction of clauses means that it 1689 01:23:25,580 --> 01:23:29,680 is an and of individual clauses, each of which has in it. 1690 01:23:29,680 --> 01:23:30,830 So something like this. 1691 01:23:30,830 --> 01:23:38,000 A or B or C, and D or not E, and F or G. Everything in parentheses 1692 01:23:38,000 --> 01:23:39,670 is one clause. 1693 01:23:39,670 --> 01:23:43,250 All of the clauses are connected to each other using an and, 1694 01:23:43,250 --> 01:23:47,400 and everything in the clause is separated using an or. 1695 01:23:47,400 --> 01:23:50,960 And this is just a standard form that we can translate a logical sentence 1696 01:23:50,960 --> 01:23:54,740 into that just makes it easy to work with and easy to manipulate. 1697 01:23:54,740 --> 01:23:57,650 And it turns out that we can take any sentence in logic 1698 01:23:57,650 --> 01:24:00,440 and turn it into conjunctive normal form, 1699 01:24:00,440 --> 01:24:04,350 just by applying some inference rules and transformations to it. 1700 01:24:04,350 --> 01:24:07,470 So we'll take a look at how we can actually do that. 1701 01:24:07,470 --> 01:24:10,280 So what is the process for taking a logical formula 1702 01:24:10,280 --> 01:24:14,582 and converting it into injunctive normal form, otherwise known as CNF? 1703 01:24:14,582 --> 01:24:16,790 Well, the process looks a little something like this. 1704 01:24:16,790 --> 01:24:19,100 We need to take all of the symbols that are not 1705 01:24:19,100 --> 01:24:22,350 part of conjunctive normal form-- the biconditionals, and the implications, 1706 01:24:22,350 --> 01:24:23,310 and so forth. 1707 01:24:23,310 --> 01:24:26,330 And turn them into something that is more closely 1708 01:24:26,330 --> 01:24:28,440 like conjunctive normal form. 1709 01:24:28,440 --> 01:24:31,040 So the first step will be to eliminate biconditionals. 1710 01:24:31,040 --> 01:24:33,440 Those if and only if double arrows. 1711 01:24:33,440 --> 01:24:35,420 And we know how to eliminate biconditionals 1712 01:24:35,420 --> 01:24:38,490 because we saw there was an inference rule to do just that. 1713 01:24:38,490 --> 01:24:42,710 Anytime I have an expression, like alpha, if and only if beta, 1714 01:24:42,710 --> 01:24:47,960 I can turn that into alpha implies beta, and beta implies alpha, based 1715 01:24:47,960 --> 01:24:50,780 on that inference rule we saw before. 1716 01:24:50,780 --> 01:24:53,150 Likewise, in addition to eliminating biconditionals, 1717 01:24:53,150 --> 01:24:55,160 I can eliminate implications as well. 1718 01:24:55,160 --> 01:24:56,980 The if/then arrows. 1719 01:24:56,980 --> 01:25:00,410 And I can do that using the same inference rule we saw before, too. 1720 01:25:00,410 --> 01:25:02,900 Taking alpha implies beta, and turning that 1721 01:25:02,900 --> 01:25:06,920 into not alpha or beta, because that is logically 1722 01:25:06,920 --> 01:25:10,730 equivalent to this first thing here. 1723 01:25:10,730 --> 01:25:13,550 Then we can move nots inwards, because we don't want nots 1724 01:25:13,550 --> 01:25:15,090 on the outsides of our expressions. 1725 01:25:15,090 --> 01:25:18,590 Conjunctive normal form requires that its just clause and clause 1726 01:25:18,590 --> 01:25:20,120 and clause and clause. 1727 01:25:20,120 --> 01:25:23,840 Any nots need to be immediately next to propositional symbols. 1728 01:25:23,840 --> 01:25:26,840 But we can move those nots around using De Morgan's laws. 1729 01:25:26,840 --> 01:25:33,290 By taking something like, not A and B, and turn it into not A or not B, 1730 01:25:33,290 --> 01:25:36,110 for example, using de Morgan's laws to manipulate that. 1731 01:25:36,110 --> 01:25:38,767 And after that, all we'll be left with are ands and ors, 1732 01:25:38,767 --> 01:25:40,100 and those are easy to deal with. 1733 01:25:40,100 --> 01:25:44,420 We can use the distributive law to distribute the ors so that the ors end 1734 01:25:44,420 --> 01:25:47,060 up on the inside of the expression, so to speak, 1735 01:25:47,060 --> 01:25:49,732 and the ands end up on the outside. 1736 01:25:49,732 --> 01:25:52,190 So this is the general pattern for how we'll take a formula 1737 01:25:52,190 --> 01:25:54,570 and convert it into conjunctive normal form. 1738 01:25:54,570 --> 01:25:56,750 And let's now take a look at an example of how 1739 01:25:56,750 --> 01:25:59,720 we would do this, and explore then, why it is that we 1740 01:25:59,720 --> 01:26:01,950 would want to do something like this. 1741 01:26:01,950 --> 01:26:03,000 Here's how we can do it. 1742 01:26:03,000 --> 01:26:04,880 Let's take this formula, for example. 1743 01:26:04,880 --> 01:26:08,510 P or Q implies R, and I'd like to convert this 1744 01:26:08,510 --> 01:26:12,650 into conjunctive normal form, where it's all ands of clauses, 1745 01:26:12,650 --> 01:26:15,090 and every clause is a disjunctive clause. 1746 01:26:15,090 --> 01:26:16,695 It's or is together. 1747 01:26:16,695 --> 01:26:18,320 So what's the first thing I need to do? 1748 01:26:18,320 --> 01:26:20,130 Well, this is an implication. 1749 01:26:20,130 --> 01:26:22,460 So let me go ahead and remove that implication. 1750 01:26:22,460 --> 01:26:27,740 Using the implication inference rule, I can turn P or Q into-- 1751 01:26:27,740 --> 01:26:34,190 P or Q implies R, into not P or Q or R. So that's the first step. 1752 01:26:34,190 --> 01:26:36,390 I've gotten rid of the implication. 1753 01:26:36,390 --> 01:26:40,370 And next, I can get rid of the not on the outside of this expression, too. 1754 01:26:40,370 --> 01:26:45,830 I can move the nots inwards so they're closer to the literals themselves 1755 01:26:45,830 --> 01:26:47,390 by using De Morgan's laws. 1756 01:26:47,390 --> 01:26:51,470 And De Morgan's law says that not P or Q is equivalent 1757 01:26:51,470 --> 01:26:57,200 to not P and not Q. Again here, just applying the inference rules 1758 01:26:57,200 --> 01:27:01,400 that we've already seen in order to translate these statements. 1759 01:27:01,400 --> 01:27:04,460 And now I have two things that are separated by an 1760 01:27:04,460 --> 01:27:07,500 or, where this thing on the inside is an and. 1761 01:27:07,500 --> 01:27:10,850 What I'd really like is to move the or so the ors are on the inside, 1762 01:27:10,850 --> 01:27:14,960 because conjunctive normal form means I need clause and clause and clause 1763 01:27:14,960 --> 01:27:15,960 and clause. 1764 01:27:15,960 --> 01:27:18,560 And so to do that, I can use the distributive law. 1765 01:27:18,560 --> 01:27:25,400 If I have not P and not Q or R, I can distribute the or R to both of these 1766 01:27:25,400 --> 01:27:31,100 to get not P or R, and not Q or R using the distributive law. 1767 01:27:31,100 --> 01:27:34,820 And this now, here at the bottom, is in conjunctive normal form. 1768 01:27:34,820 --> 01:27:39,080 It is a conjunction, an and, of disjunctions, 1769 01:27:39,080 --> 01:27:42,510 of clauses that just are separated by ors. 1770 01:27:42,510 --> 01:27:46,400 So this process can be used by any formula to take a logical sentence 1771 01:27:46,400 --> 01:27:49,220 and turn it into this conjuncture normal form, where 1772 01:27:49,220 --> 01:27:54,022 I have clause and clause and clause and clause and clause, and so on. 1773 01:27:54,022 --> 01:27:54,980 So why is this helpful? 1774 01:27:54,980 --> 01:27:57,260 Why do we even care about taking all these sentences 1775 01:27:57,260 --> 01:27:58,940 and converting them into this form? 1776 01:27:58,940 --> 01:28:02,860 It's because once they're in this form where we have these clauses, 1777 01:28:02,860 --> 01:28:06,850 these clauses are the inputs to the resolution, inference rule that we 1778 01:28:06,850 --> 01:28:07,660 saw a moment ago. 1779 01:28:07,660 --> 01:28:11,260 That if I have two clauses where there's something that conflicts, 1780 01:28:11,260 --> 01:28:13,690 or something complementary between those two clauses, 1781 01:28:13,690 --> 01:28:17,470 I can resolve them to get a new clause, to draw a new conclusion. 1782 01:28:17,470 --> 01:28:20,530 And we call this process inference by resolution. 1783 01:28:20,530 --> 01:28:23,920 Using the resolution rule to draw some sort of inference. 1784 01:28:23,920 --> 01:28:28,020 And it's based on the same idea, that if I have P or Q, this clause, 1785 01:28:28,020 --> 01:28:32,680 and I have not P or R, that I can resolve these two clauses together 1786 01:28:32,680 --> 01:28:35,680 to get Q or R as the resulting clause. 1787 01:28:35,680 --> 01:28:39,243 A new piece of information that I didn't have before. 1788 01:28:39,243 --> 01:28:42,160 Now, a couple of key points that are worth noting about this before we 1789 01:28:42,160 --> 01:28:43,990 talk about the actual algorithm. 1790 01:28:43,990 --> 01:28:47,850 One thing is that, let's imagine we have P or Q or S, 1791 01:28:47,850 --> 01:28:52,780 and I also have not P or R or S. The resolution rule says 1792 01:28:52,780 --> 01:28:55,990 that because this P conflicts with this not P, 1793 01:28:55,990 --> 01:29:01,052 we would resolve to put everything else together, to get Q or S or R or S. 1794 01:29:01,052 --> 01:29:05,800 But it turns out that this double S is redundant-- or S here and or S there. 1795 01:29:05,800 --> 01:29:07,990 It doesn't change the meaning of the sentence. 1796 01:29:07,990 --> 01:29:10,570 So in resolution, when we do this resolution process, 1797 01:29:10,570 --> 01:29:13,180 we usually also do a process known as factoring, 1798 01:29:13,180 --> 01:29:16,850 where we take any duplicate variables that show up and just eliminate them. 1799 01:29:16,850 --> 01:29:23,170 So Q or S or R or S just becomes Q or R or S. The S only needs to appear once. 1800 01:29:23,170 --> 01:29:26,290 No need to include it multiple times. 1801 01:29:26,290 --> 01:29:28,420 Now, one final question worth considering 1802 01:29:28,420 --> 01:29:33,250 is what happens if I try to resolve P and not P together? 1803 01:29:33,250 --> 01:29:36,760 If I know that P is true, and I know that not P is true, 1804 01:29:36,760 --> 01:29:39,850 well, resolution says I can merge these clauses together and look 1805 01:29:39,850 --> 01:29:41,570 at everything else. 1806 01:29:41,570 --> 01:29:43,600 Well, in this case, there is nothing else. 1807 01:29:43,600 --> 01:29:46,570 So I'm left with what we might call the empty clause. 1808 01:29:46,570 --> 01:29:48,130 I'm left with nothing. 1809 01:29:48,130 --> 01:29:51,220 And the empty clause is always false. 1810 01:29:51,220 --> 01:29:54,220 The empty clause is equivalent to just being false. 1811 01:29:54,220 --> 01:29:55,720 And that's pretty reasonable. 1812 01:29:55,720 --> 01:30:01,690 Because it's impossible for both P and not P to both hold at the same time. 1813 01:30:01,690 --> 01:30:03,910 P is either true or it's not true. 1814 01:30:03,910 --> 01:30:07,250 Which means that if P is true, then this must be false. 1815 01:30:07,250 --> 01:30:09,410 And if this is true, then this must be false. 1816 01:30:09,410 --> 01:30:12,170 There's no way for both of these to hold at the same time. 1817 01:30:12,170 --> 01:30:15,640 So if ever I try and resolve these two, it's a contradiction, 1818 01:30:15,640 --> 01:30:18,820 and I'll end up getting this empty clause, where the empty clause 1819 01:30:18,820 --> 01:30:21,730 I can call equivalent to false. 1820 01:30:21,730 --> 01:30:25,750 And this idea that if I resolve these two contradictory terms I 1821 01:30:25,750 --> 01:30:30,280 get the empty clause, this is the basis for our inference by resolution 1822 01:30:30,280 --> 01:30:30,937 algorithm. 1823 01:30:30,937 --> 01:30:32,770 Here is how we're going to perform inference 1824 01:30:32,770 --> 01:30:35,420 by resolution at a very high level. 1825 01:30:35,420 --> 01:30:40,000 We want to prove that our knowledge base entails some query alpha. 1826 01:30:40,000 --> 01:30:43,300 That based on the knowledge we have, we can prove conclusively 1827 01:30:43,300 --> 01:30:45,950 that alpha is going to be true. 1828 01:30:45,950 --> 01:30:47,540 How are we going to do that? 1829 01:30:47,540 --> 01:30:49,450 Well, in order to do that, we're going to try 1830 01:30:49,450 --> 01:30:54,100 to prove that if we know the knowledge and not alpha, that that 1831 01:30:54,100 --> 01:30:55,760 would be a contradiction. 1832 01:30:55,760 --> 01:30:58,510 And this is a common technique in computer science more generally. 1833 01:30:58,510 --> 01:31:01,720 This idea of proving something by contradiction. 1834 01:31:01,720 --> 01:31:04,510 If I want to prove that something is true, 1835 01:31:04,510 --> 01:31:08,320 I can do so by first assuming that it is false, 1836 01:31:08,320 --> 01:31:10,450 and showing that it would be contradictory. 1837 01:31:10,450 --> 01:31:12,770 Showing that it leads to some contradiction. 1838 01:31:12,770 --> 01:31:16,120 And if the thing I'm trying to prove, if when I assume it's false 1839 01:31:16,120 --> 01:31:19,090 leads to a contradiction, then it must be true. 1840 01:31:19,090 --> 01:31:22,745 And that's the logical approach, or the idea, behind a proof by contradiction. 1841 01:31:22,745 --> 01:31:24,370 And that's what we're going to do here. 1842 01:31:24,370 --> 01:31:27,610 We want to prove that this query alpha is true, 1843 01:31:27,610 --> 01:31:30,440 so we're going to assume that it's not true. 1844 01:31:30,440 --> 01:31:32,410 We're going to assume not alpha. 1845 01:31:32,410 --> 01:31:34,960 And we're going to try and prove that it's a contradiction. 1846 01:31:34,960 --> 01:31:37,240 If we do get a contradiction, well, then we 1847 01:31:37,240 --> 01:31:40,690 know that our knowledge entails the query alpha. 1848 01:31:40,690 --> 01:31:43,300 If we don't get a contradiction, there is no entail. 1849 01:31:43,300 --> 01:31:45,670 This is this idea of a proof by contradiction 1850 01:31:45,670 --> 01:31:48,280 of assuming the opposite of what you're trying to prove, 1851 01:31:48,280 --> 01:31:51,940 and if you can demonstrate that that's a contradiction, then what you're proving 1852 01:31:51,940 --> 01:31:54,043 must be true. 1853 01:31:54,043 --> 01:31:55,960 But more formally, how do we actually do this? 1854 01:31:55,960 --> 01:32:00,460 How do we check that knowledge base and not alpha 1855 01:32:00,460 --> 01:32:02,420 is going to lead to a contradiction? 1856 01:32:02,420 --> 01:32:05,620 Well, here is where resolution comes into play. 1857 01:32:05,620 --> 01:32:09,490 To determine if our knowledge base entails some query alpha, 1858 01:32:09,490 --> 01:32:13,510 we're going to convert knowledge base and not alpha to conjunction 1859 01:32:13,510 --> 01:32:14,260 normal form. 1860 01:32:14,260 --> 01:32:18,700 That form where we have a whole bunch of clauses that are all anded together. 1861 01:32:18,700 --> 01:32:21,080 And when we have these individual clauses, 1862 01:32:21,080 --> 01:32:25,900 now we can keep checking to see if we can use resolution 1863 01:32:25,900 --> 01:32:27,940 to produce a new clause. 1864 01:32:27,940 --> 01:32:31,030 We can take any pair of clauses and check. 1865 01:32:31,030 --> 01:32:34,240 Is there some literal that is the opposite of each other, 1866 01:32:34,240 --> 01:32:36,430 or complementary to each other, in both of them? 1867 01:32:36,430 --> 01:32:40,180 For example, I have a P in one clause, and a not P in another clause. 1868 01:32:40,180 --> 01:32:43,780 Or an R in one clause, and a not R in another clause. 1869 01:32:43,780 --> 01:32:45,940 If ever I have that situation, where once I 1870 01:32:45,940 --> 01:32:49,210 convert to conjunctive normal form and I have a whole bunch of clauses, 1871 01:32:49,210 --> 01:32:54,010 I see two clauses that I can resolve to produce a new clause, then I'll do so. 1872 01:32:54,010 --> 01:32:55,340 This process occurs in a loop. 1873 01:32:55,340 --> 01:32:58,240 I'm going to keep checking to see if I can use resolution 1874 01:32:58,240 --> 01:33:01,820 to produce a new clause, and keep using those new clauses to try to generate 1875 01:33:01,820 --> 01:33:04,820 more new clauses after that. 1876 01:33:04,820 --> 01:33:07,310 Now, it just so may happen that eventually, we 1877 01:33:07,310 --> 01:33:09,593 may produce the empty clause. 1878 01:33:09,593 --> 01:33:11,260 The clause we were talking about before. 1879 01:33:11,260 --> 01:33:16,010 If I resolve P and not P together, that produces the empty clause. 1880 01:33:16,010 --> 01:33:18,920 And the empty clause we know to be false. 1881 01:33:18,920 --> 01:33:21,920 Because we know that there's no way for both P and not 1882 01:33:21,920 --> 01:33:25,590 P to both simultaneously be true. 1883 01:33:25,590 --> 01:33:29,395 So if ever we produce the empty clause, then we have a contradiction. 1884 01:33:29,395 --> 01:33:31,520 And if we have a contradiction, that's exactly what 1885 01:33:31,520 --> 01:33:33,983 we were trying to do in a proof by contradiction. 1886 01:33:33,983 --> 01:33:36,650 If we have a contradiction, then we know that our knowledge base 1887 01:33:36,650 --> 01:33:38,690 must entail this query alpha. 1888 01:33:38,690 --> 01:33:41,890 We know that alpha must be true. 1889 01:33:41,890 --> 01:33:44,240 And it turns out-- and we won't go into the proof here-- 1890 01:33:44,240 --> 01:33:48,050 but you can show that, otherwise, if you don't produce the empty clause, 1891 01:33:48,050 --> 01:33:49,700 then there is no entailment. 1892 01:33:49,700 --> 01:33:52,970 If we run into a situation where there are no more new clauses to add, 1893 01:33:52,970 --> 01:33:55,280 we've done all the resolution that we can do, 1894 01:33:55,280 --> 01:33:57,680 and yet we still haven't produced the empty clause, 1895 01:33:57,680 --> 01:34:00,680 then there is no entailment in this case. 1896 01:34:00,680 --> 01:34:03,020 And this now is the resolution algorithm. 1897 01:34:03,020 --> 01:34:04,940 And it's very abstract looking, especially 1898 01:34:04,940 --> 01:34:07,862 this idea of what does that even mean to have the empty clause. 1899 01:34:07,862 --> 01:34:09,320 So let's take a look at an example. 1900 01:34:09,320 --> 01:34:13,250 Actually try and prove some entailment by using this inference 1901 01:34:13,250 --> 01:34:15,720 by resolution process. 1902 01:34:15,720 --> 01:34:16,970 So here's our question. 1903 01:34:16,970 --> 01:34:18,500 We have this knowledge base. 1904 01:34:18,500 --> 01:34:20,240 Here is the knowledge that we know. 1905 01:34:20,240 --> 01:34:27,560 A or B, and not B or C, and not C. And we want to know if all of this 1906 01:34:27,560 --> 01:34:30,140 entails A. 1907 01:34:30,140 --> 01:34:32,870 So this is our knowledge base here, this whole log thing. 1908 01:34:32,870 --> 01:34:37,430 And our query alpha is just this propositional symbol A. 1909 01:34:37,430 --> 01:34:38,510 So what do we do? 1910 01:34:38,510 --> 01:34:40,760 Well, first we want to prove by contradiction. 1911 01:34:40,760 --> 01:34:43,880 So we want to first assume that A is false, 1912 01:34:43,880 --> 01:34:46,360 and see if that leads to some sort of contradiction. 1913 01:34:46,360 --> 01:34:48,110 So here is what we're going to start with. 1914 01:34:48,110 --> 01:34:52,970 A or B, and not B or C, and not C. This is our knowledge base. 1915 01:34:52,970 --> 01:34:55,550 And we're going to assume not A. We're going 1916 01:34:55,550 --> 01:35:01,050 to assume that the thing we're trying to prove is, in fact, false. 1917 01:35:01,050 --> 01:35:03,800 And so this is now in conjunctive normal form, 1918 01:35:03,800 --> 01:35:05,690 and I have four different clauses. 1919 01:35:05,690 --> 01:35:12,930 I have A or B. I have not B or C. I have not C. And I have not A. 1920 01:35:12,930 --> 01:35:17,630 And now, I can begin to just pick two clauses that I can resolve and apply 1921 01:35:17,630 --> 01:35:20,090 the resolution rule to them. 1922 01:35:20,090 --> 01:35:21,890 And so looking at these four clauses I see, 1923 01:35:21,890 --> 01:35:25,730 all right, these two clauses are ones I can resolve. 1924 01:35:25,730 --> 01:35:29,570 I can resolve them because there are complementary literals that 1925 01:35:29,570 --> 01:35:30,300 show up in them. 1926 01:35:30,300 --> 01:35:32,870 There's a C here, and a not C here. 1927 01:35:32,870 --> 01:35:38,640 So just looking at these two clauses, if I know that not B or C is true, 1928 01:35:38,640 --> 01:35:41,210 and I know that C is not true, well, then 1929 01:35:41,210 --> 01:35:45,690 I can resolve these two clauses to say, all right, not B. That must be true. 1930 01:35:45,690 --> 01:35:49,340 I can generate this new clause as a new piece of information 1931 01:35:49,340 --> 01:35:51,995 that I now know to be true. 1932 01:35:51,995 --> 01:35:53,870 And all right, now I can repeat this process. 1933 01:35:53,870 --> 01:35:55,100 Do the process again. 1934 01:35:55,100 --> 01:35:58,460 Can I use resolution again to get some new conclusion? 1935 01:35:58,460 --> 01:35:59,510 Well, it turns out I can. 1936 01:35:59,510 --> 01:36:03,020 I can use that new clause I just generated, along with this one here. 1937 01:36:03,020 --> 01:36:04,910 There are complementary literals. 1938 01:36:04,910 --> 01:36:10,580 This B is complementary to, or conflicts with this not B over here. 1939 01:36:10,580 --> 01:36:16,640 And so if I know that A or B is true, and I know that B is not true, 1940 01:36:16,640 --> 01:36:19,860 well, then the only remaining possibility is that A must be true. 1941 01:36:19,860 --> 01:36:23,940 So now we have A. That is a new clause that I've been able to generate. 1942 01:36:23,940 --> 01:36:25,940 And now I can do this one more time, and looking 1943 01:36:25,940 --> 01:36:27,530 for two clauses that can be resolved. 1944 01:36:27,530 --> 01:36:29,780 And you might programmatically do this by just looping 1945 01:36:29,780 --> 01:36:32,600 over all possible pairs of clauses and checking 1946 01:36:32,600 --> 01:36:34,520 for complementary literals in each. 1947 01:36:34,520 --> 01:36:36,830 And here I can say, all right, I found two clauses. 1948 01:36:36,830 --> 01:36:40,740 Not A and A that conflict with each other. 1949 01:36:40,740 --> 01:36:43,070 And when I resolve these two together, well, this 1950 01:36:43,070 --> 01:36:46,340 is the same as when we were resolving P and not P from before. 1951 01:36:46,340 --> 01:36:50,060 When I resolve these two clauses together, I get rid of the As, 1952 01:36:50,060 --> 01:36:52,550 and I'm left with the empty clause. 1953 01:36:52,550 --> 01:36:55,070 And the empty clause we know to be false, which means we 1954 01:36:55,070 --> 01:36:56,210 have a contradiction. 1955 01:36:56,210 --> 01:36:58,970 Which means we can safely say, that this whole knowledge 1956 01:36:58,970 --> 01:37:02,420 base does entail A. That if this sentence is true, 1957 01:37:02,420 --> 01:37:06,480 that we know that A, for sure, is also true. 1958 01:37:06,480 --> 01:37:09,020 So this now, using inference by resolution, 1959 01:37:09,020 --> 01:37:12,050 is an entirely different way to take some statement 1960 01:37:12,050 --> 01:37:14,540 and try and prove that it is, in fact, true. 1961 01:37:14,540 --> 01:37:16,880 Instead of enumerating all of the possible worlds 1962 01:37:16,880 --> 01:37:20,150 that we might be in in order to try to figure out in which case 1963 01:37:20,150 --> 01:37:23,060 is the knowledge base true, and in which case is our query true. 1964 01:37:23,060 --> 01:37:26,270 Instead we use this resolution algorithm to say, 1965 01:37:26,270 --> 01:37:29,360 let's keep trying to figure out what conclusions we can draw 1966 01:37:29,360 --> 01:37:31,450 and see if we reach a contradiction. 1967 01:37:31,450 --> 01:37:33,200 And if we reach a contradiction, then that 1968 01:37:33,200 --> 01:37:37,310 tells us something about whether our knowledge actually entails the query 1969 01:37:37,310 --> 01:37:38,110 or not. 1970 01:37:38,110 --> 01:37:40,485 And it turns out there are many different algorithms that 1971 01:37:40,485 --> 01:37:41,745 can be used for inference. 1972 01:37:41,745 --> 01:37:44,120 What we've just looked at here are just a couple of them. 1973 01:37:44,120 --> 01:37:48,380 And, in fact, all of this is just based on one particular type of logic. 1974 01:37:48,380 --> 01:37:52,190 It's based on propositional logic, where we have these individual symbols 1975 01:37:52,190 --> 01:37:55,610 and we connect them using and, and or, and not, and implies, 1976 01:37:55,610 --> 01:37:56,930 and biconditionals. 1977 01:37:56,930 --> 01:38:01,170 But propositional logic is not the only kind of logic that exists. 1978 01:38:01,170 --> 01:38:03,170 And in fact, we see that there are limitations 1979 01:38:03,170 --> 01:38:05,960 that exist in propositional logic, especially 1980 01:38:05,960 --> 01:38:10,310 as we saw in examples like with the Mastermind example, 1981 01:38:10,310 --> 01:38:12,740 or with the example with the logic puzzle 1982 01:38:12,740 --> 01:38:16,550 where we had different Hogwart's has people that belong to different houses, 1983 01:38:16,550 --> 01:38:19,370 and we were trying to figure out who belonged to which houses. 1984 01:38:19,370 --> 01:38:21,530 There were a lot of different propositional symbols 1985 01:38:21,530 --> 01:38:25,867 that we needed in order to represent some fairly basic ideas. 1986 01:38:25,867 --> 01:38:27,950 So now as a final topic that we'll take a look at, 1987 01:38:27,950 --> 01:38:31,040 just before we end class today, is one final type of logic, 1988 01:38:31,040 --> 01:38:33,200 different from propositional logic, known 1989 01:38:33,200 --> 01:38:35,330 as first order logic, which is a little bit more 1990 01:38:35,330 --> 01:38:37,790 powerful than propositional logic, and is 1991 01:38:37,790 --> 01:38:41,540 going to make it easier for us to express certain types of ideas. 1992 01:38:41,540 --> 01:38:44,090 In propositional logic, if we think back to that puzzle 1993 01:38:44,090 --> 01:38:46,280 with the people and the Hogwart's houses, 1994 01:38:46,280 --> 01:38:48,620 we had a whole bunch of symbols, and every symbol 1995 01:38:48,620 --> 01:38:50,510 could only be true or false. 1996 01:38:50,510 --> 01:38:53,120 We had a symbol for Minerva Gryffindor, which was either true 1997 01:38:53,120 --> 01:38:55,580 if Minerva was in Gryffindor, and false otherwise. 1998 01:38:55,580 --> 01:38:58,310 And likewise, for Minerva Hufflepuff, and Minerva Ravenclaw, 1999 01:38:58,310 --> 01:39:01,230 and Minerva Slytherin, and so forth. 2000 01:39:01,230 --> 01:39:03,170 But this was starting to get quite redundant. 2001 01:39:03,170 --> 01:39:05,150 That we wanted some way to be able to express 2002 01:39:05,150 --> 01:39:07,817 that there's a relationship between these propositional symbols. 2003 01:39:07,817 --> 01:39:09,725 That Minerva shows up in all of them. 2004 01:39:09,725 --> 01:39:11,600 And also, I would have liked to have not have 2005 01:39:11,600 --> 01:39:14,270 had so many different symbols to represent 2006 01:39:14,270 --> 01:39:17,718 what really was a fairly straightforward problem. 2007 01:39:17,718 --> 01:39:19,760 So first order logic will give us a different way 2008 01:39:19,760 --> 01:39:21,890 of trying to deal with this idea by giving us 2009 01:39:21,890 --> 01:39:23,810 two different types of symbols. 2010 01:39:23,810 --> 01:39:27,320 We're going to have constant symbols that are going to represent objects, 2011 01:39:27,320 --> 01:39:29,150 like people or houses. 2012 01:39:29,150 --> 01:39:31,730 And then predicate symbols, which you can 2013 01:39:31,730 --> 01:39:35,870 think of as relations or functions, that take an input and evaluate them to, 2014 01:39:35,870 --> 01:39:37,520 like, true or false, for example. 2015 01:39:37,520 --> 01:39:41,720 That tell us whether or not some property of some constant, 2016 01:39:41,720 --> 01:39:45,358 or some pair of constants, or multiple constants, actually holds. 2017 01:39:45,358 --> 01:39:47,400 So we'll see an example of that in just a moment. 2018 01:39:47,400 --> 01:39:52,302 But for now, in this same problem, our constant symbols might be objects. 2019 01:39:52,302 --> 01:39:53,510 Things like people or houses. 2020 01:39:53,510 --> 01:39:55,820 So Minerva, Pomona, Horace, Gildaroy. 2021 01:39:55,820 --> 01:39:57,740 Those are all constant symbols. 2022 01:39:57,740 --> 01:39:58,820 As are my four houses-- 2023 01:39:58,820 --> 01:40:02,300 Gryffindor, Hufflepuff, Ravenclaw, and Slytherin. 2024 01:40:02,300 --> 01:40:04,670 Predicates, meanwhile, these predicate symbols 2025 01:40:04,670 --> 01:40:08,180 are going to be properties that might hold true or false 2026 01:40:08,180 --> 01:40:10,400 of these individual constants. 2027 01:40:10,400 --> 01:40:14,990 So person might hold true of Minerva, but it would be false for Gryffindor, 2028 01:40:14,990 --> 01:40:16,610 because Gryffindor is not a person. 2029 01:40:16,610 --> 01:40:19,580 And house is going to hold true for Ravenclaw, 2030 01:40:19,580 --> 01:40:21,920 but it's not going to hold true for Horace, for example, 2031 01:40:21,920 --> 01:40:24,110 because Horace is a person. 2032 01:40:24,110 --> 01:40:27,740 And belongs, meanwhile, is going to be some relation that is 2033 01:40:27,740 --> 01:40:30,560 going to relate people to their houses. 2034 01:40:30,560 --> 01:40:34,760 And it's going to only tell me when someone belongs to a house or does not. 2035 01:40:34,760 --> 01:40:39,380 So let's take a look at some examples of what a sentence in first order logic 2036 01:40:39,380 --> 01:40:40,662 might actually look like. 2037 01:40:40,662 --> 01:40:42,620 A sentence might look like something like this. 2038 01:40:42,620 --> 01:40:46,010 Person Minerva, with Minerva in parentheses. 2039 01:40:46,010 --> 01:40:50,180 And person being a predicate symbol, Minerva being a constant symbol. 2040 01:40:50,180 --> 01:40:55,010 This sentence in first order logic effectively means Minerva is a person, 2041 01:40:55,010 --> 01:40:58,808 or the person property applies to the Minerva object. 2042 01:40:58,808 --> 01:41:01,100 So if I want to say something like Minerva is a person, 2043 01:41:01,100 --> 01:41:05,090 here is how I express that idea using first order logic. 2044 01:41:05,090 --> 01:41:08,000 Meanwhile, I can say something like house Gryffindor 2045 01:41:08,000 --> 01:41:11,630 to, likewise, express the idea that Gryffindor is a house. 2046 01:41:11,630 --> 01:41:13,070 I can do that this way. 2047 01:41:13,070 --> 01:41:16,643 And all of the same logical connectives that we saw in propositional logic, 2048 01:41:16,643 --> 01:41:18,060 those are going to work here, too. 2049 01:41:18,060 --> 01:41:21,050 So and, or, implication, biconditional, not. 2050 01:41:21,050 --> 01:41:25,220 In fact, I can use not to say something like, not house Minerva. 2051 01:41:25,220 --> 01:41:29,330 And this sentence in first order logic means something like Minerva 2052 01:41:29,330 --> 01:41:30,350 is not a house. 2053 01:41:30,350 --> 01:41:35,897 It is not true that the house property applies to Minerva. 2054 01:41:35,897 --> 01:41:38,480 Meanwhile, in addition to some of these predicate symbols that 2055 01:41:38,480 --> 01:41:41,210 just take a single argument, some of our predicate symbols 2056 01:41:41,210 --> 01:41:43,580 are going to express binary relations. 2057 01:41:43,580 --> 01:41:46,500 Relations between two of its arguments. 2058 01:41:46,500 --> 01:41:50,030 So I could say something like, belongs to, and then two inputs, 2059 01:41:50,030 --> 01:41:53,480 Minerva and Gryffindor to express the idea 2060 01:41:53,480 --> 01:41:56,200 that Minerva belongs to Gryffindor. 2061 01:41:56,200 --> 01:41:59,210 And so now here's the key difference, or one of the key differences, 2062 01:41:59,210 --> 01:42:01,220 between this and propositional logic. 2063 01:42:01,220 --> 01:42:04,940 In propositional logic, I needed one symbol for Minerva Gryffindor, 2064 01:42:04,940 --> 01:42:07,580 and one symbol for Minerva Hufflepuff, and one symbol 2065 01:42:07,580 --> 01:42:10,670 for all the other people's Gryffindor and Hufflepuff variables. 2066 01:42:10,670 --> 01:42:14,840 In this case, I just need one symbol for each of my people, 2067 01:42:14,840 --> 01:42:17,870 and one symbol for each of my houses, and then I 2068 01:42:17,870 --> 01:42:21,230 can express, as a predicate, something like, belongs to, 2069 01:42:21,230 --> 01:42:24,980 and say, belongs to Minerva Gryffindor to express the idea 2070 01:42:24,980 --> 01:42:27,740 that Minerva belongs to Gryffindor house. 2071 01:42:27,740 --> 01:42:30,080 So already we can see that first order logic 2072 01:42:30,080 --> 01:42:35,030 is quite expressive in being able to express these sorts of sentences using 2073 01:42:35,030 --> 01:42:37,430 the existing constant symbols and predicates that 2074 01:42:37,430 --> 01:42:40,550 already exist, while minimizing the number of new symbols 2075 01:42:40,550 --> 01:42:41,520 that I need to create. 2076 01:42:41,520 --> 01:42:45,560 I can just use eight symbols for people, for houses, instead 2077 01:42:45,560 --> 01:42:50,360 of 16 symbols for every possible combination of each. 2078 01:42:50,360 --> 01:42:53,300 But first order logic gives us a couple of additional features 2079 01:42:53,300 --> 01:42:56,300 that we can use to express even more complex ideas. 2080 01:42:56,300 --> 01:43:00,470 And these additional features are generally known as quantifiers. 2081 01:43:00,470 --> 01:43:03,110 And there are two main quantifiers in first order logic. 2082 01:43:03,110 --> 01:43:05,930 The first of which is universal quantification. 2083 01:43:05,930 --> 01:43:09,110 Universal quantification lets me express an idea, 2084 01:43:09,110 --> 01:43:13,300 like something is going to be true for all values of a variable. 2085 01:43:13,300 --> 01:43:17,970 Like for all values of x, some statement is going to hold true. 2086 01:43:17,970 --> 01:43:20,900 So what might a sentence in universal quantification look like? 2087 01:43:20,900 --> 01:43:25,370 Well, we're going to use this upside-down A to mean for all. 2088 01:43:25,370 --> 01:43:31,080 So upside-down Ax means for all values of x, where x is any object, 2089 01:43:31,080 --> 01:43:33,140 this is going to hold true. 2090 01:43:33,140 --> 01:43:41,120 Belongs to x Gryffindor implies not belongs to x Hufflepuff. 2091 01:43:41,120 --> 01:43:42,470 So let's try and parse this out. 2092 01:43:42,470 --> 01:43:46,790 This means that for all values of x, If this holds true, 2093 01:43:46,790 --> 01:43:51,200 if x belongs to Gryffindor, then this does not hold true. 2094 01:43:51,200 --> 01:43:54,500 X does not belong to Hufflepuff. 2095 01:43:54,500 --> 01:43:56,930 So translated into English, this sentence 2096 01:43:56,930 --> 01:44:01,610 is saying something like, for all objects x, if x belongs to Gryffindor, 2097 01:44:01,610 --> 01:44:05,090 then x does not belong to Hufflepuff, for example. 2098 01:44:05,090 --> 01:44:09,530 Or phrased even more simply, anyone in Gryffindor is not a Hufflepuff. 2099 01:44:09,530 --> 01:44:12,360 A simplified way of saying the same thing. 2100 01:44:12,360 --> 01:44:15,620 So this universal quantification lets us express an idea, 2101 01:44:15,620 --> 01:44:20,750 like something is going to hold true for all values of a particular variable. 2102 01:44:20,750 --> 01:44:23,270 In addition to universal quantification, though, we also 2103 01:44:23,270 --> 01:44:26,240 have existential quantification. 2104 01:44:26,240 --> 01:44:28,730 Whereas universal quantification said that something 2105 01:44:28,730 --> 01:44:31,640 is going to be true for all values of variable, 2106 01:44:31,640 --> 01:44:34,700 existential quantification says that some expression 2107 01:44:34,700 --> 01:44:37,910 is going to be true for some value of a variable. 2108 01:44:37,910 --> 01:44:40,980 At least one value of the variable. 2109 01:44:40,980 --> 01:44:43,070 So let's take a look at a sample sentence using 2110 01:44:43,070 --> 01:44:44,870 existential quantification. 2111 01:44:44,870 --> 01:44:46,890 One such sentence looks like this. 2112 01:44:46,890 --> 01:44:50,760 There exists an x-- this backwards E stands for exists. 2113 01:44:50,760 --> 01:44:54,980 And here we're saying there exists an x, such that house x 2114 01:44:54,980 --> 01:44:57,680 and belongs to Minerva x. 2115 01:44:57,680 --> 01:45:01,760 In other words, there exists some object x, where x is a house, 2116 01:45:01,760 --> 01:45:04,790 and Minerva belongs to x. 2117 01:45:04,790 --> 01:45:07,970 Or phrased a little more succinctly in English, you're just saying, 2118 01:45:07,970 --> 01:45:09,710 Minerva belongs to a house. 2119 01:45:09,710 --> 01:45:14,600 There's some object that is a house, and Minerva belongs to a house. 2120 01:45:14,600 --> 01:45:17,600 And combining this universal and existential quantification, 2121 01:45:17,600 --> 01:45:20,570 we can create far more sophisticated logical statements 2122 01:45:20,570 --> 01:45:23,630 than we were able to just using propositional logic. 2123 01:45:23,630 --> 01:45:26,150 I could combine these to say something like this. 2124 01:45:26,150 --> 01:45:33,170 For all x, person x implies there exists a y, such that house y 2125 01:45:33,170 --> 01:45:35,525 and belongs to xy. 2126 01:45:35,525 --> 01:45:36,900 So a lot of stuff going on there. 2127 01:45:36,900 --> 01:45:37,910 A lot of symbols. 2128 01:45:37,910 --> 01:45:40,610 Let's try and parse it out and just understand what it's saying. 2129 01:45:40,610 --> 01:45:47,400 Here we're saying that for all values of x, if x is a person, then this is true. 2130 01:45:47,400 --> 01:45:49,940 So in other words, I'm saying for all people, 2131 01:45:49,940 --> 01:45:53,270 and we call that person x, this statement is going to be true. 2132 01:45:53,270 --> 01:45:55,100 What statement is true of all people? 2133 01:45:55,100 --> 01:46:00,050 Well, there exists a y that is the house, so there exists some house, 2134 01:46:00,050 --> 01:46:03,050 and x belongs to y. 2135 01:46:03,050 --> 01:46:05,840 In other words I'm saying, that for all people out there, 2136 01:46:05,840 --> 01:46:11,760 there exists some house, such that x, the person, belongs to y, the house. 2137 01:46:11,760 --> 01:46:15,830 So say it more succinctly, I'm saying that every person belongs to a house. 2138 01:46:15,830 --> 01:46:21,470 That for all x, if x is a person, then there exists a house that x belongs to. 2139 01:46:21,470 --> 01:46:25,030 And so we can now express a lot more powerful ideas using this idea now 2140 01:46:25,030 --> 01:46:26,030 of first order logic. 2141 01:46:26,030 --> 01:46:28,655 And it turns out there are many other kinds of logic out there. 2142 01:46:28,655 --> 01:46:31,820 There's second order logic and other higher order logic, each of which 2143 01:46:31,820 --> 01:46:35,000 allows us to express more and more complex ideas. 2144 01:46:35,000 --> 01:46:38,630 But all of it, in this case, is really in pursuit of the same goal, which 2145 01:46:38,630 --> 01:46:40,550 is the representation of knowledge. 2146 01:46:40,550 --> 01:46:44,060 We want our AI agents to be able to know information, 2147 01:46:44,060 --> 01:46:46,160 to represent that information, whether that's 2148 01:46:46,160 --> 01:46:49,730 using propositional logic, or first order logic, or some other logic. 2149 01:46:49,730 --> 01:46:51,680 And then be able to reason based on that. 2150 01:46:51,680 --> 01:46:53,360 To be able to draw conclusions. 2151 01:46:53,360 --> 01:46:54,230 Make inferences. 2152 01:46:54,230 --> 01:46:57,080 Figure out whether there's some sort of entailment relationship, 2153 01:46:57,080 --> 01:46:59,780 as by using some sort of inference algorithm. 2154 01:46:59,780 --> 01:47:02,772 Something like inference by resolution, or model checking, 2155 01:47:02,772 --> 01:47:04,730 or any number of these other algorithms that we 2156 01:47:04,730 --> 01:47:07,700 can use in order to take information that we know 2157 01:47:07,700 --> 01:47:10,280 and translate it to additional conclusions. 2158 01:47:10,280 --> 01:47:13,040 So all of this has helped us to create AI 2159 01:47:13,040 --> 01:47:15,980 that is able to represent information about what it knows 2160 01:47:15,980 --> 01:47:17,860 and what it doesn't know. 2161 01:47:17,860 --> 01:47:19,610 Next time though, we'll take a look at how 2162 01:47:19,610 --> 01:47:23,150 we can make our AI even more powerful by not just encoding information 2163 01:47:23,150 --> 01:47:26,690 that we know for sure to be true, and not to be true, but also, 2164 01:47:26,690 --> 01:47:28,110 to take a look at uncertainty. 2165 01:47:28,110 --> 01:47:31,550 To look at what happens if AI thinks that something might be probable, 2166 01:47:31,550 --> 01:47:35,810 or maybe not very probable, or somewhere in between those two extremes, 2167 01:47:35,810 --> 01:47:40,310 all in the pursuit of trying to build our intelligent systems to be even more 2168 01:47:40,310 --> 01:47:41,180 intelligent. 2169 01:47:41,180 --> 01:47:43,210 We'll see you next time.