1
00:00:00,000 --> 00:00:06,480

2
00:00:06,480 --> 00:00:09,240
>> BRIAN SCASSELLATI: Welcome
to the CS50 AI video series.

3
00:00:09,240 --> 00:00:10,500
My name is Scas.

4
00:00:10,500 --> 00:00:14,510
And today, we're going to be
talking about self-driving cars.

5
00:00:14,510 --> 00:00:17,490
Now, I'm a little bit
embarrassed to admit that when

6
00:00:17,490 --> 00:00:24,760
I was a kid a large part of my childhood
revolved around David Hasselhoff.

7
00:00:24,760 --> 00:00:31,060
Now, this was before he was involved
in anything like America's Got Talent,

8
00:00:31,060 --> 00:00:37,170
or involved as a lifeguard in Baywatch,
or even before he had sort of history

9
00:00:37,170 --> 00:00:40,620
as a pop star in Germany.

10
00:00:40,620 --> 00:00:44,550
>> In my childhood, David Hasselhoff
was the supporting actor

11
00:00:44,550 --> 00:00:48,590
on a well-known television
series called Knight Rider.

12
00:00:48,590 --> 00:00:53,730
And I say he was the supporting actor,
because really the star of this show

13
00:00:53,730 --> 00:00:58,594
was a car, a Trans Am named
Kit, who could drive by itself.

14
00:00:58,594 --> 00:00:59,260
Kit was amazing.

15
00:00:59,260 --> 00:01:01,100
It could talk to you.

16
00:01:01,100 --> 00:01:02,300
>> It could solve problems.

17
00:01:02,300 --> 00:01:03,920
It could drive all over the place.

18
00:01:03,920 --> 00:01:05,459
It also had lasers and rockets.

19
00:01:05,459 --> 00:01:08,520
So it was a fantastic
vehicle to start with.

20
00:01:08,520 --> 00:01:11,900
>> But this was the science
fiction of the time,

21
00:01:11,900 --> 00:01:14,230
autonomous cars that
could make decisions,

22
00:01:14,230 --> 00:01:16,460
that could drive along the road.

23
00:01:16,460 --> 00:01:19,610
And at every point in
one of these episodes,

24
00:01:19,610 --> 00:01:22,080
David Hasselhoff would
get in trouble and the car

25
00:01:22,080 --> 00:01:25,300
would start driving and go and save him.

26
00:01:25,300 --> 00:01:28,000
That was our science fiction.

27
00:01:28,000 --> 00:01:31,480
>> Even in just the last 30
years, that science fiction

28
00:01:31,480 --> 00:01:35,160
has gone from on television
and on the screen

29
00:01:35,160 --> 00:01:38,840
into reality, into commercial products.

30
00:01:38,840 --> 00:01:43,380
Today, we're going to talk about how it
is that autonomous vehicles are really

31
00:01:43,380 --> 00:01:49,540
able to go and drive and to
absolutely amazing things.

32
00:01:49,540 --> 00:01:51,430
But let's start with the history.

33
00:01:51,430 --> 00:01:54,840
>> Because these self-driving cars
didn't come out of nowhere.

34
00:01:54,840 --> 00:01:57,310
And in fact, the very
first self-driving cars,

35
00:01:57,310 --> 00:02:00,420
the really first influential
research projects,

36
00:02:00,420 --> 00:02:03,230
came out of a project called Navlab.

37
00:02:03,230 --> 00:02:07,020
Navlab was a project that spanned
almost two decades at Carnegie Mellon

38
00:02:07,020 --> 00:02:08,340
University.

39
00:02:08,340 --> 00:02:11,570
>> And they built a variety
of different vehicles

40
00:02:11,570 --> 00:02:14,940
that started out looking
like small minivans, that

41
00:02:14,940 --> 00:02:20,630
were Humvees, that were eventually just
sedans, and minivans, even city buses.

42
00:02:20,630 --> 00:02:23,900
And these different devices
have sensors in them,

43
00:02:23,900 --> 00:02:26,490
had computational
systems put into them, so

44
00:02:26,490 --> 00:02:31,170
that they could steer, and brake,
and accelerate autonomously

45
00:02:31,170 --> 00:02:33,350
all on their own.

46
00:02:33,350 --> 00:02:37,080
Now, these systems were
very primitive at the start.

47
00:02:37,080 --> 00:02:41,590
>> And they relied upon very
specific lane markings.

48
00:02:41,590 --> 00:02:47,530
So the most impressive system that
they had built was part of Navlab five.

49
00:02:47,530 --> 00:02:51,420
And in 1995, this
vehicle, it was minivan,

50
00:02:51,420 --> 00:02:56,560
drove from Pittsburgh to San
Diego, almost 3,000 miles.

51
00:02:56,560 --> 00:03:00,850
And 98% of the time, the only
thing controlling the steering

52
00:03:00,850 --> 00:03:02,670
was the computer.

53
00:03:02,670 --> 00:03:05,870
>> 98% of the time, it was
completely autonomous

54
00:03:05,870 --> 00:03:08,816
as it drove almost from coast to coast.

55
00:03:08,816 --> 00:03:10,870
Now, that's incredibly impressive.

56
00:03:10,870 --> 00:03:14,490
And when we think about that,
there's a lot for us to consider.

57
00:03:14,490 --> 00:03:16,250
>> What kinds of sensors was it using?

58
00:03:16,250 --> 00:03:19,580
What kind of decision
was it really making?

59
00:03:19,580 --> 00:03:23,310
What was that other 2% of the time?

60
00:03:23,310 --> 00:03:26,050
>> We're going to try to get to
some of these issues today.

61
00:03:26,050 --> 00:03:28,410
And as we look at them,
we're going to try

62
00:03:28,410 --> 00:03:31,870
to uncover what it is that
the structure underneath

63
00:03:31,870 --> 00:03:37,440
and how the computation is directly
driving these applications.

64
00:03:37,440 --> 00:03:40,190
Now, Navlab was a tremendous success.

65
00:03:40,190 --> 00:03:44,220
And it was the basis of all of our
modern thoughts about autonomous

66
00:03:44,220 --> 00:03:46,010
vehicles.

67
00:03:46,010 --> 00:03:51,660
>> But self-driving cars didn't really
become well-known and popular

68
00:03:51,660 --> 00:03:58,060
until in 2005 DARPA, as part of
their grand challenge effort,

69
00:03:58,060 --> 00:04:03,420
put together a program to try
to build an autonomous vehicle.

70
00:04:03,420 --> 00:04:04,940
And they made this a challenge.

71
00:04:04,940 --> 00:04:08,370
They put a $2 million prize
out there with the idea

72
00:04:08,370 --> 00:04:11,740
that some really smart research
team would come along and be

73
00:04:11,740 --> 00:04:15,190
able to claim this $2 million prize.

74
00:04:15,190 --> 00:04:17,600
>> Now, the prize wasn't
going to be easy to get.

75
00:04:17,600 --> 00:04:19,970
In order to get the
prize, you would have

76
00:04:19,970 --> 00:04:23,500
to build a vehicle that
with no human intervention

77
00:04:23,500 --> 00:04:28,310
was able to drive 150 mile
course through the rough terrain

78
00:04:28,310 --> 00:04:30,340
of the desert.

79
00:04:30,340 --> 00:04:31,860
And that was a lot task.

80
00:04:31,860 --> 00:04:36,830
>> And at the time, people thought that
they were really crazy in doing this.

81
00:04:36,830 --> 00:04:42,740
So the first meeting of the
grand challenge was in 2004.

82
00:04:42,740 --> 00:04:48,680
And out of that 150 mile course, the
farthest, the best team that they had,

83
00:04:48,680 --> 00:04:53,880
went about 11 miles before
the system failed completely.

84
00:04:53,880 --> 00:04:57,290
>> Now, to give you an idea
of how difficult that is,

85
00:04:57,290 --> 00:05:02,770
that 11 miles was seen as an
absolute phenomenal success.

86
00:05:02,770 --> 00:05:04,290
But DARPA wanted more.

87
00:05:04,290 --> 00:05:07,120
And so they offered this same
prize the following year.

88
00:05:07,120 --> 00:05:09,830
>> And just one year later,
the technology had

89
00:05:09,830 --> 00:05:15,270
advanced to the point where not just one
system was able to complete the course,

90
00:05:15,270 --> 00:05:19,290
but five different robot
cars finished that course.

91
00:05:19,290 --> 00:05:24,800
The fastest one finished 132
miles in under seven hours.

92
00:05:24,800 --> 00:05:27,070
That was a robot called Stanley.

93
00:05:27,070 --> 00:05:30,810
>> Stanley was built by the
Stanford racing team.

94
00:05:30,810 --> 00:05:33,150
And as you can see at
the top, it had a number

95
00:05:33,150 --> 00:05:37,050
of different sensors up on the
hood, up on the top of the vehicle,

96
00:05:37,050 --> 00:05:38,980
and all throughout.

97
00:05:38,980 --> 00:05:44,020
Using a combination of cameras,
infrared, and regular light,

98
00:05:44,020 --> 00:05:49,250
using radar and sonar systems on board,
using laser range finders to detect

99
00:05:49,250 --> 00:05:54,740
obstacles, this vehicle was able
to navigate over very rough terrain

100
00:05:54,740 --> 00:05:59,170
autonomously steering,
autonomously breaking, autonomously

101
00:05:59,170 --> 00:06:02,120
applying the gas.

102
00:06:02,120 --> 00:06:05,930
>> That was a real achievement.

103
00:06:05,930 --> 00:06:10,300
Today, we see this happening
even as a grander scale.

104
00:06:10,300 --> 00:06:13,670
Many of you have heard of
the Google self-driving car.

105
00:06:13,670 --> 00:06:21,180
And these vehicles have logged over 1.2
million miles in the last few years,

106
00:06:21,180 --> 00:06:24,110
no human intervention whatsoever.

107
00:06:24,110 --> 00:06:26,150
>> In fact, every time
that the Google car has

108
00:06:26,150 --> 00:06:30,140
been involved in any kind of accident,
it's either been because it was parked,

109
00:06:30,140 --> 00:06:33,195
or because some human was
so interested in what it

110
00:06:33,195 --> 00:06:36,860
was doing that they ran into the car.

111
00:06:36,860 --> 00:06:42,160
So with all of these systems,
we see this complexity emerging.

112
00:06:42,160 --> 00:06:45,380
And in this very short
period of time, we've

113
00:06:45,380 --> 00:06:50,120
gone from the realm of science
fiction to commercial reality.

114
00:06:50,120 --> 00:06:52,470
>> So let's start to take
these systems apart.

115
00:06:52,470 --> 00:06:57,120
Let's try to understand how it is that
they work, what do they actually doing.

116
00:06:57,120 --> 00:06:59,490
To do that, we're going to
use the same kinds of skills

117
00:06:59,490 --> 00:07:01,240
that we've talked about in class.

118
00:07:01,240 --> 00:07:04,030
Whenever you see a problem,
what we're going to try to do

119
00:07:04,030 --> 00:07:05,600
is to try to decompose it.

120
00:07:05,600 --> 00:07:08,680
>> Start with the simplest
form that we can.

121
00:07:08,680 --> 00:07:12,110
And then build outward
from that simple form.

122
00:07:12,110 --> 00:07:15,950
So that leads us to the question, what
is the simplest form of autonomous

123
00:07:15,950 --> 00:07:16,890
driving?

124
00:07:16,890 --> 00:07:21,430
At what point is a computer
actually in control of my car?

125
00:07:21,430 --> 00:07:24,020
>> Now, the answer may surprise you there.

126
00:07:24,020 --> 00:07:28,920
Because almost every vehicle sold
today in the US or Europe or anywhere

127
00:07:28,920 --> 00:07:33,110
actually is partially
an autonomous vehicle.

128
00:07:33,110 --> 00:07:39,660
Using systems like anti-lock brakes,
these systems are really autonomous.

129
00:07:39,660 --> 00:07:44,610
>> That is, when I step on the break,
what I'm doing is I'm asking the car

130
00:07:44,610 --> 00:07:46,800
please break now.

131
00:07:46,800 --> 00:07:49,600
I'm not actually directly
stepping on something that

132
00:07:49,600 --> 00:07:52,710
applies the brake pad to the rotor.

133
00:07:52,710 --> 00:07:57,580
And the whole point of anti-lock brakes
is that at some point along the way

134
00:07:57,580 --> 00:08:00,610
I'll be able to press down on the break.

135
00:08:00,610 --> 00:08:04,620
>> But the car will recognize
that the wheel is slipping.

136
00:08:04,620 --> 00:08:10,670
And it will throttle that break signal,
so that the brake doesn't lock up.

137
00:08:10,670 --> 00:08:16,100
These anti-lock brake systems are,
in a way, making decisions for you.

138
00:08:16,100 --> 00:08:19,610
>> And really, they're the ones that
are in charge of the braking system.

139
00:08:19,610 --> 00:08:22,070
You're making a request.

140
00:08:22,070 --> 00:08:25,120
But you're not actually in control.

141
00:08:25,120 --> 00:08:30,070
>> So we could try to recognize this and
break it down into the component parts.

142
00:08:30,070 --> 00:08:33,900
And we could think about it as
a little bit of pseudocode code.

143
00:08:33,900 --> 00:08:36,760
That is, while I'm
stepping on the brake,

144
00:08:36,760 --> 00:08:40,400
while I'm applying pressure
to the brake pedal,

145
00:08:40,400 --> 00:08:44,110
anti-lock brake system
is checking continuously

146
00:08:44,110 --> 00:08:48,480
to see is each of these wheels slipping.

147
00:08:48,480 --> 00:08:51,740
>> And using some internal
sensors within the car,

148
00:08:51,740 --> 00:08:54,020
they're detecting whether
or not the wheel is actually

149
00:08:54,020 --> 00:08:56,350
stopping or whether it's sliding.

150
00:08:56,350 --> 00:09:01,890
And if it's sliding, the anti-lock
brake systems disengages the brakes,

151
00:09:01,890 --> 00:09:03,190
and then lets it go.

152
00:09:03,190 --> 00:09:07,750
And when the wheels stop
sliding, it reapplies them.

153
00:09:07,750 --> 00:09:09,850
>> That is, I'm making a request.

154
00:09:09,850 --> 00:09:11,900
I'm stepping on the brake.

155
00:09:11,900 --> 00:09:17,540
But the actual breaking is being
decided by this small piece of software.

156
00:09:17,540 --> 00:09:23,320
So really, all of our cars are
already autonomous vehicles.

157
00:09:23,320 --> 00:09:27,470
>> Now, that's not what we think about
when we think about autonomous vehicles.

158
00:09:27,470 --> 00:09:31,090
We think about cars where I can
take my hands away from the wheel,

159
00:09:31,090 --> 00:09:34,500
and we can just let it go.

160
00:09:34,500 --> 00:09:38,910
Now, that's not happening on a
grand scale everywhere yet today.

161
00:09:38,910 --> 00:09:40,990
>> But there are pieces of
that that are starting

162
00:09:40,990 --> 00:09:44,960
to come into the commercial sector.

163
00:09:44,960 --> 00:09:52,120
Since 2003, Toyota, and following
that many other manufacturers,

164
00:09:52,120 --> 00:09:55,740
everyone from Ford and
Lincoln to Mercedes Benz,

165
00:09:55,740 --> 00:09:59,660
has been offering some type
of intelligent parking assist.

166
00:09:59,660 --> 00:10:02,890
That is, there are sensors
in the car, typically

167
00:10:02,890 --> 00:10:07,010
ultrasonic sensors for short
range detection of obstacles,

168
00:10:07,010 --> 00:10:12,360
that are able to recognize where
there are cars, vehicles, people,

169
00:10:12,360 --> 00:10:16,230
any type of obstacle around the vehicle.

170
00:10:16,230 --> 00:10:22,070
>> You then press a button on the dash
and ask the car, please park now.

171
00:10:22,070 --> 00:10:24,460
You issue a request.

172
00:10:24,460 --> 00:10:28,570
The autonomous system then takes
over and using those sensors

173
00:10:28,570 --> 00:10:33,250
is able to guide the car into
a particular parking position.

174
00:10:33,250 --> 00:10:36,260
In some of these models, there's
a parallel parking version

175
00:10:36,260 --> 00:10:39,980
and a backing into a spot version.

176
00:10:39,980 --> 00:10:46,200
>> And each of these different applications
evokes a different piece of software.

177
00:10:46,200 --> 00:10:48,145
Now, that software
isn't anything strange

178
00:10:48,145 --> 00:10:51,820
or isn't anything that you
can't understand at this point.

179
00:10:51,820 --> 00:10:54,520
It's just following
these sensor signals.

180
00:10:54,520 --> 00:10:57,340
>> If there's something to
close on the left-hand side

181
00:10:57,340 --> 00:11:00,570
and I have space on the right,
then I'll steer a little bit,

182
00:11:00,570 --> 00:11:03,010
so that I can move over to the right.

183
00:11:03,010 --> 00:11:06,840
Many of the early parking systems
would control the steering angle,

184
00:11:06,840 --> 00:11:11,420
but require the user, the human driver,
to actually step on the accelerator

185
00:11:11,420 --> 00:11:13,480
or step on the break.

186
00:11:13,480 --> 00:11:18,040
More modern systems actually control
that completely by themselves.

187
00:11:18,040 --> 00:11:21,190
>> So for example, in a Mercedes
S Class vehicle right

188
00:11:21,190 --> 00:11:25,410
now, you can pull alongside where
you'd like to park, press a button,

189
00:11:25,410 --> 00:11:28,610
and it will parallel park for you
without your hands on the wheel

190
00:11:28,610 --> 00:11:31,770
or your feet on the pedals.

191
00:11:31,770 --> 00:11:36,140
Now all of these systems
rely upon the sensors

192
00:11:36,140 --> 00:11:39,510
that they are building
into these vehicles today.

193
00:11:39,510 --> 00:11:44,950
And whether we use those sensors
for detecting potential obstacles

194
00:11:44,950 --> 00:11:49,580
and alerting the user or whether we use
those sensors to detect an obstacle,

195
00:11:49,580 --> 00:11:54,730
and then automatically steer away,
that's just a matter of software.

196
00:11:54,730 --> 00:11:58,320
>> In fact, just a few
weeks ago, Tesla, who's

197
00:11:58,320 --> 00:12:03,680
been building fantastic vehicles with
all of these sensors in them for years

198
00:12:03,680 --> 00:12:07,870
now, issued a software update.

199
00:12:07,870 --> 00:12:12,522
And that software update allowed
the vehicles for the first time

200
00:12:12,522 --> 00:12:17,370
to enter an automatic driving
more, an auto pilot they called it.

201
00:12:17,370 --> 00:12:21,490
This auto pilot allowed the vehicle
to detect collisions and automatically

202
00:12:21,490 --> 00:12:26,600
break, to follow another vehicle
that's in front of it, matching speed,

203
00:12:26,600 --> 00:12:32,450
to stay within the lanes, to look with
cameras, both infrared and visible

204
00:12:32,450 --> 00:12:34,980
light, and to be able
to say whether or not

205
00:12:34,980 --> 00:12:38,930
you're drifting out of your lane or
not, adjust the steering appropriately,

206
00:12:38,930 --> 00:12:42,380
and even to change lanes
when the user signals.

207
00:12:42,380 --> 00:12:46,240
>> All of these different features were
just a matter of a software update.

208
00:12:46,240 --> 00:12:48,610
That is, all of these
users woke up one morning

209
00:12:48,610 --> 00:12:52,910
to find this new software
available in their vehicles.

210
00:12:52,910 --> 00:12:56,210
Because the sensor systems
were already there.

211
00:12:56,210 --> 00:13:00,730
>> Now, in all of these cases,
we're seeing these software

212
00:13:00,730 --> 00:13:03,870
based systems becoming
more and more prevalent.

213
00:13:03,870 --> 00:13:06,500
They're out there in
commercial products already.

214
00:13:06,500 --> 00:13:10,350
And the future is that we're
going to see more of that.

215
00:13:10,350 --> 00:13:13,880
>> In fact, just this
year, Freightliner was

216
00:13:13,880 --> 00:13:18,810
able to unveil an autonomous truck,
an autonomous tractor trailer,

217
00:13:18,810 --> 00:13:22,650
that they're testing legally
on the road in Nevada.

218
00:13:22,650 --> 00:13:25,960
These vehicles, again,
follow a predetermined route.

219
00:13:25,960 --> 00:13:27,570
They stay within their lane.

220
00:13:27,570 --> 00:13:30,280
>> They accelerate and
decelerate in response

221
00:13:30,280 --> 00:13:33,790
to obstacles or traffic conditions.

222
00:13:33,790 --> 00:13:38,780
And they even obey some of the
other niceties of the road.

223
00:13:38,780 --> 00:13:43,250
All of these systems are
becoming more and more complex.

224
00:13:43,250 --> 00:13:45,260
>> But they're still not quite autonomous.

225
00:13:45,260 --> 00:13:47,610
They're still not
quite doing everything.

226
00:13:47,610 --> 00:13:49,810
That is, they're still
requiring a human driver

227
00:13:49,810 --> 00:13:54,030
to be present to make
some high level decisions.

228
00:13:54,030 --> 00:13:57,500
>> And one of the things that we're
going to see in the next five years

229
00:13:57,500 --> 00:14:00,930
are a variety of legal
and ethical questions

230
00:14:00,930 --> 00:14:07,500
that revolve around the software
being built for these driverless cars.

231
00:14:07,500 --> 00:14:10,830
How is it that a driverless
car should respond if it's

232
00:14:10,830 --> 00:14:13,320
surrounded by a group of people?

233
00:14:13,320 --> 00:14:16,360
What happens if the driverless
car is skidding on the road

234
00:14:16,360 --> 00:14:21,710
and you can steer towards a crowd
of 10 people or a crowd of 7 people?

235
00:14:21,710 --> 00:14:24,420
What should the car do?

236
00:14:24,420 --> 00:14:29,780
>> In all of these cases, there's a rich
variety of questions to be asked.

237
00:14:29,780 --> 00:14:33,610
And they are not just software
questions, legal questions,

238
00:14:33,610 --> 00:14:36,520
ethical questions,
philosophical questions.

239
00:14:36,520 --> 00:14:40,600
And they are ones that we as a
community will have to address.

240
00:14:40,600 --> 00:14:44,500
>> So I'll leave you with one last
thought, this one from Randall Munroe,

241
00:14:44,500 --> 00:14:48,640
of XKCD, one of my favorite comics.

242
00:14:48,640 --> 00:14:53,430
It's not just that we're going to see
these vehicles being built and being

243
00:14:53,430 --> 00:14:55,000
designed with software.

244
00:14:55,000 --> 00:14:58,440
But we're going to see people
try to exploit them as well.

245
00:14:58,440 --> 00:15:03,850
>> How is it going to be when someone
can, over Wi-Fi, download a patch

246
00:15:03,850 --> 00:15:07,720
or upload a virus to your car?

247
00:15:07,720 --> 00:15:10,770
What kinds of things will happen then?

248
00:15:10,770 --> 00:15:13,860
This one's a little bit
more playful of an example.

249
00:15:13,860 --> 00:15:17,510
But these are the questions
we're going to deal with soon.

250
00:15:17,510 --> 00:15:18,930
>> Thanks for joining me.

251
00:15:18,930 --> 00:15:20,420
I hope you've enjoyed it.

252
00:15:20,420 --> 00:15:22,650
And we'll see you next time.

253
00:15:22,650 --> 00:15:23,294