1
00:00:00,000 --> 00:00:02,880
[AUDIO LOGO]

2
00:00:02,880 --> 00:00:04,800


3
00:00:04,800 --> 00:00:06,450
CARTER ZENKE: Well, hello one and all.

4
00:00:06,450 --> 00:00:09,040
And welcome to our short on pytest.

5
00:00:09,040 --> 00:00:13,110
Now, pytest is a module you can use
to test your code more thoroughly

6
00:00:13,110 --> 00:00:15,280
than you could on your own.

7
00:00:15,280 --> 00:00:18,300
And I have here a program
called convert.py,

8
00:00:18,300 --> 00:00:23,100
whose purpose is to convert this unit
called an astronomical unit, often used

9
00:00:23,100 --> 00:00:27,660
in space exploration and so on, into
a more friendly measurement called

10
00:00:27,660 --> 00:00:28,980
a meter.

11
00:00:28,980 --> 00:00:31,890
So I have here down below
the convert function.

12
00:00:31,890 --> 00:00:33,850
And this does the following.

13
00:00:33,850 --> 00:00:40,500
It first checks if the argument
au is of type int or float.

14
00:00:40,500 --> 00:00:45,430
And if it is not, it raises, in this
case, a TypeError for the wrong type.

15
00:00:45,430 --> 00:00:47,890
It says "au must be an int or float."

16
00:00:47,890 --> 00:00:49,620
In this case, if it is not.

17
00:00:49,620 --> 00:00:53,640
Otherwise, if au is
of type int or float,

18
00:00:53,640 --> 00:00:57,630
well, we'll go ahead and return the
conversion between astronomical units

19
00:00:57,630 --> 00:01:01,110
and meters from this function convert.

20
00:01:01,110 --> 00:01:05,950
Now, I could go ahead and
run Python convert.py,

21
00:01:05,950 --> 00:01:07,970
and I could maybe test this on my own.

22
00:01:07,970 --> 00:01:11,350
I could type in maybe
1 for AU and get back,

23
00:01:11,350 --> 00:01:13,850
in this case, some
large number of meters.

24
00:01:13,850 --> 00:01:17,350
I can go ahead and type python
convert.py and get back, in this case,

25
00:01:17,350 --> 00:01:19,430
again, some large number of meters.

26
00:01:19,430 --> 00:01:22,537
But I could probably be
more thorough than this.

27
00:01:22,537 --> 00:01:24,370
And I would really hate
to be the programmer

28
00:01:24,370 --> 00:01:26,680
to sit down and write these
tests myself just kind

29
00:01:26,680 --> 00:01:30,010
of hitting Control-L
to close my terminal,

30
00:01:30,010 --> 00:01:34,130
run this program again, type in some
new input, and check the result by hand.

31
00:01:34,130 --> 00:01:37,370
So that's why things like
pytest exist in the world.

32
00:01:37,370 --> 00:01:42,580
Now, if I want to write some tests for
my code and do so with code itself,

33
00:01:42,580 --> 00:01:46,330
I can use by convention,
per pytest, a file

34
00:01:46,330 --> 00:01:49,730
called test underscore
and then some given name.

35
00:01:49,730 --> 00:01:56,080
So I'll call this file
test_convert.py to test, in this case,

36
00:01:56,080 --> 00:01:58,360
this function I have called convert.

37
00:01:58,360 --> 00:02:04,150
And again, by convention, in pytest,
files that hold tests for our code,

38
00:02:04,150 --> 00:02:07,300
in this case, have to begin with test_.

39
00:02:07,300 --> 00:02:11,270
So I'll go ahead and open
up code of test_convert.py.

40
00:02:11,270 --> 00:02:16,840
And again, by convention, I'll go ahead
and import this module called pytest.

41
00:02:16,840 --> 00:02:18,740
Now, what could I do?

42
00:02:18,740 --> 00:02:23,230
Well, I want to, in this case,
test my convert function.

43
00:02:23,230 --> 00:02:28,060
In general, when we're using py
test, we're engaging in unit testing,

44
00:02:28,060 --> 00:02:31,730
testing individual units or
functions in our program.

45
00:02:31,730 --> 00:02:34,490
So I'll save main for testing later on.

46
00:02:34,490 --> 00:02:37,060
I want to focus on convert right now.

47
00:02:37,060 --> 00:02:43,880
So I could go ahead and import the
convert function from my convert file.

48
00:02:43,880 --> 00:02:49,310
I could type from convert, this file
here, import the convert function.

49
00:02:49,310 --> 00:02:54,760
And now I can use the convert
function inside of test_convert.py.

50
00:02:54,760 --> 00:03:00,550
Well, here, again by convention, if I
want to write some code to test convert,

51
00:03:00,550 --> 00:03:05,670
I should write it inside of a
function that begins with test_.

52
00:03:05,670 --> 00:03:09,270
So maybe the simplest thing to test
here is just some simple values,

53
00:03:09,270 --> 00:03:13,690
like the conversion between
1 AU and 2 meters instead.

54
00:03:13,690 --> 00:03:18,550
So I'll type def and then
test_conversion here.

55
00:03:18,550 --> 00:03:21,390
And this is making for
myself a new function

56
00:03:21,390 --> 00:03:25,410
that will run when I call
pytest in my terminal.

57
00:03:25,410 --> 00:03:28,500
As we've seen before,
pytest relies on this idea

58
00:03:28,500 --> 00:03:30,780
of asserting something to be true.

59
00:03:30,780 --> 00:03:34,050
And if it is not, pytest will
let us know if a program has not

60
00:03:34,050 --> 00:03:36,190
behaved as we expect.

61
00:03:36,190 --> 00:03:41,400
So here, I could assert, let's say,
that when I call convert and give it

62
00:03:41,400 --> 00:03:45,030
an input of 1, that should
be equal to, in this case,

63
00:03:45,030 --> 00:03:46,360
this large number of meters.

64
00:03:46,360 --> 00:03:54,270
I'll go ahead and type
this in here, 149597870700.

65
00:03:54,270 --> 00:04:00,450
Let me double check
this here, 149597870700.

66
00:04:00,450 --> 00:04:03,000
So this is a large number
of meters, but it is

67
00:04:03,000 --> 00:04:07,310
the conversion between 1 AU to meters.

68
00:04:07,310 --> 00:04:10,420
Now, a side note here is that
if I was running these tests,

69
00:04:10,420 --> 00:04:14,060
I'd probably want to be very sure
this is the right conversion.

70
00:04:14,060 --> 00:04:16,149
This is probably a good
chance to say if you

71
00:04:16,149 --> 00:04:20,773
had to type out-- what is this, 1,
2, 3, 6, 9, almost 12 numbers here,

72
00:04:20,773 --> 00:04:22,690
just be doubly sure
you're typing each of them

73
00:04:22,690 --> 00:04:24,710
right when you're
writing tests like these.

74
00:04:24,710 --> 00:04:29,570
So here, I'll go ahead and run
pytest now that I have, in this case,

75
00:04:29,570 --> 00:04:31,970
my test_convert.py file.

76
00:04:31,970 --> 00:04:35,320
And when I run pytest in
my terminal, it will look

77
00:04:35,320 --> 00:04:39,130
for a file like this, test_convert--

78
00:04:39,130 --> 00:04:41,080
or begins with test_.

79
00:04:41,080 --> 00:04:46,390
It will then run all of the functions
that begin with test_ and let me know

80
00:04:46,390 --> 00:04:49,640
how many have passed and how
many have, in this case, failed.

81
00:04:49,640 --> 00:04:51,640
So I'll go ahead and run pytest.

82
00:04:51,640 --> 00:04:55,240
And fingers crossed, we
see that one has passed.

83
00:04:55,240 --> 00:04:58,580
In test_convert.py, there
is one little dot here,

84
00:04:58,580 --> 00:05:03,040
meaning this function did what it was
supposed to do and did not encounter any

85
00:05:03,040 --> 00:05:04,120
errors.

86
00:05:04,120 --> 00:05:07,990
Now, it's a good idea to think
through other kinds of scenarios

87
00:05:07,990 --> 00:05:09,530
to test in your code.

88
00:05:09,530 --> 00:05:12,400
In this case, I'm converting 1 AU.

89
00:05:12,400 --> 00:05:15,530
But what are some other representative
scenarios we could test?

90
00:05:15,530 --> 00:05:18,910
Maybe something like
50 AU as well-- convert

91
00:05:18,910 --> 00:05:22,880
50, which happens to be this
other long string of numbers.

92
00:05:22,880 --> 00:05:26,000
So again, message here
is maybe write your test

93
00:05:26,000 --> 00:05:28,030
so you get fewer numbers than this--

94
00:05:28,030 --> 00:05:33,520
3535000.

95
00:05:33,520 --> 00:05:42,010
And I'll double check this
again, 74798935335000.

96
00:05:42,010 --> 00:05:43,750
Great.

97
00:05:43,750 --> 00:05:51,050
All right, so this is the conversion
between 50 AU to meters as well.

98
00:05:51,050 --> 00:05:53,180
I'll go ahead and run pytest again.

99
00:05:53,180 --> 00:05:57,140
And fingers crossed, we see that,
again, those tests have passed.

100
00:05:57,140 --> 00:06:01,400
And notice here, it still says
"1 passed" down below, 1 passed.

101
00:06:01,400 --> 00:06:04,170
We count, in this case, by the function.

102
00:06:04,170 --> 00:06:07,340
But in this case, these
two individual tests

103
00:06:07,340 --> 00:06:12,320
have to both pass for this test
to be considered passing as well.

104
00:06:12,320 --> 00:06:14,780
Now, we could think of
other kinds of scenarios

105
00:06:14,780 --> 00:06:18,540
to test, in this case, negative
numbers, maybe 0 as well.

106
00:06:18,540 --> 00:06:21,260
Things like that are really
useful for testing your code here.

107
00:06:21,260 --> 00:06:23,490
But more on unit testing in lecture.

108
00:06:23,490 --> 00:06:25,980
We'll focus here mostly on pytest.

109
00:06:25,980 --> 00:06:27,800
So here, what else could we do?

110
00:06:27,800 --> 00:06:30,740
Well, we've seen that in convert.py--

111
00:06:30,740 --> 00:06:36,470
that in convert.py, I have, in
this case, not just the pathway

112
00:06:36,470 --> 00:06:39,500
of returning the actual
conversion but also

113
00:06:39,500 --> 00:06:42,090
this pathway of raising a type error.

114
00:06:42,090 --> 00:06:46,890
And it's, again, a good idea to test all
these pathways through our functions.

115
00:06:46,890 --> 00:06:51,620
So we could ideally try to test,
does convert raise a TypeError

116
00:06:51,620 --> 00:06:55,070
when given an input of the wrong type?

117
00:06:55,070 --> 00:06:57,420
Well, thanks to pytest,
we can actually do this.

118
00:06:57,420 --> 00:06:59,660
I'll go ahead and go
back to test_convert.

119
00:06:59,660 --> 00:07:06,170
And let's go ahead and try to add in,
in this case, a new test, one that tests

120
00:07:06,170 --> 00:07:09,270
for the proper error being raised.

121
00:07:09,270 --> 00:07:11,640
I'll call this one test_error.

122
00:07:11,640 --> 00:07:15,950
And again, we'll make sure
this begins with test_.

123
00:07:15,950 --> 00:07:20,480
I can then use what's called
a context manager with here

124
00:07:20,480 --> 00:07:25,910
to go ahead and try to run
convert and expect, let's

125
00:07:25,910 --> 00:07:30,210
say, that I get a
TypeError, in particular.

126
00:07:30,210 --> 00:07:32,660
So by convention with
the syntax of pytest,

127
00:07:32,660 --> 00:07:39,290
I could type "with
pytest.raises TypeError" colon.

128
00:07:39,290 --> 00:07:43,010
And indented inside of
this with block, I'll

129
00:07:43,010 --> 00:07:49,310
type convert and maybe give as input
parentheses quote 1, just like this.

130
00:07:49,310 --> 00:07:55,190
So this now means that if I were to call
convert and give as input "1," quote,

131
00:07:55,190 --> 00:07:59,910
unquote, well, I would expect
a TypeError to be raised.

132
00:07:59,910 --> 00:08:04,200
So slightly different syntax
from what we see on line 7 and 8

133
00:08:04,200 --> 00:08:05,710
but the same kind of semantics.

134
00:08:05,710 --> 00:08:12,130
We expect whatever we run inside of
this block here will raise a TypeError.

135
00:08:12,130 --> 00:08:14,650
I'll go ahead and run
python convert dot--

136
00:08:14,650 --> 00:08:16,280
or actually, just pytest.

137
00:08:16,280 --> 00:08:17,320
I'll hit pytest here.

138
00:08:17,320 --> 00:08:20,380
And now we'll see that
two tests have passed, so

139
00:08:20,380 --> 00:08:24,650
not just test conversion,
but also test_error as well.

140
00:08:24,650 --> 00:08:30,160
So it seems like convert does raise
a TypeError when given in this case

141
00:08:30,160 --> 00:08:33,700
an argument that is
not an int or a float.

142
00:08:33,700 --> 00:08:35,630
Now, there's still more to test.

143
00:08:35,630 --> 00:08:41,140
In fact, up here, I was really just
testing whole numbers, 1 and 50,

144
00:08:41,140 --> 00:08:45,685
but it's worth thinking of floats as
a special case when we're testing.

145
00:08:45,685 --> 00:08:49,580
In particular, this is because
of floating point in precision.

146
00:08:49,580 --> 00:08:52,210
So you've probably heard
that there are only

147
00:08:52,210 --> 00:08:56,930
a certain number of bits we can use
to represent certain numbers in code.

148
00:08:56,930 --> 00:09:03,530
In fact, in Python, maybe have anywhere
between 16 bits to 32 bits to 64 bits.

149
00:09:03,530 --> 00:09:06,290
But however many number
of bits we have, we're

150
00:09:06,290 --> 00:09:10,650
always going to have some finite
number of bits to represent numbers.

151
00:09:10,650 --> 00:09:12,590
When it comes to floats,
there are actually

152
00:09:12,590 --> 00:09:15,380
an infinite number of
floating point values out

153
00:09:15,380 --> 00:09:18,960
in the real world, any
number of real numbers.

154
00:09:18,960 --> 00:09:24,290
And so at some point, we actually can't
represent decimals completely precisely.

155
00:09:24,290 --> 00:09:28,550
And so when testing them, when testing
inputs and outputs and testing floats

156
00:09:28,550 --> 00:09:30,860
in particular, we have to
make room for something

157
00:09:30,860 --> 00:09:35,330
called tolerance or approximation,
some bit of wiggle room to say that,

158
00:09:35,330 --> 00:09:41,840
yes, this value here is 1.000000001,
but it's close enough to 1

159
00:09:41,840 --> 00:09:44,000
that we'll consider it as 1.

160
00:09:44,000 --> 00:09:47,210
Let's go ahead and try and
see what kind of utilities

161
00:09:47,210 --> 00:09:49,610
pytest has for those kinds of scenarios.

162
00:09:49,610 --> 00:09:52,640
I'll go ahead and look
at a new test here,

163
00:09:52,640 --> 00:09:57,560
one called def test_float_conversion.

164
00:09:57,560 --> 00:10:01,220
And for consistency, I'll say
this is testing our integer

165
00:10:01,220 --> 00:10:04,190
conversion between whole
numbers and whole numbers.

166
00:10:04,190 --> 00:10:07,580
But this here is testing
our float conversion.

167
00:10:07,580 --> 00:10:11,450
And here, I'll try to
assert that I will be

168
00:10:11,450 --> 00:10:18,650
able to convert 0.001, a pretty
small float itself, to some value

169
00:10:18,650 --> 00:10:25,760
like this, 149597870.691.

170
00:10:25,760 --> 00:10:31,660
Double check this here, 149597870.691.

171
00:10:31,660 --> 00:10:35,808
So here, I'm essentially
saying I expect that 0.001,

172
00:10:35,808 --> 00:10:39,470
what we have as input
to convert, will be

173
00:10:39,470 --> 00:10:43,640
equal to this particular
floating point value here.

174
00:10:43,640 --> 00:10:48,110
But this might not always be the case
depending on the level of precision

175
00:10:48,110 --> 00:10:49,680
I'm expecting.

176
00:10:49,680 --> 00:10:53,360
So if I want to allow for
some kind of tolerance,

177
00:10:53,360 --> 00:11:02,150
let's say I'll accept this number,
0.690000001 as also a valid equivalency

178
00:11:02,150 --> 00:11:06,010
here, I can use pytest.approx--

179
00:11:06,010 --> 00:11:11,970
pytest.approx and give as input
the value I'm hoping to compare.

180
00:11:11,970 --> 00:11:17,730
And this will use pytest's sensible
defaults to allow for some tolerance

181
00:11:17,730 --> 00:11:22,590
when testing the equivalency of
these two floating point values,

182
00:11:22,590 --> 00:11:27,030
both from the return value of convert
and this one I've written over here.

183
00:11:27,030 --> 00:11:31,680
So if I run pytest again, I'll
see that three of these tests

184
00:11:31,680 --> 00:11:35,170
have passed, which
seems pretty good here.

185
00:11:35,170 --> 00:11:39,180
But let's play around a little
bit with the actual tolerance

186
00:11:39,180 --> 00:11:41,680
that approx gives us here.

187
00:11:41,680 --> 00:11:45,000
I could, if I wanted to,
change the tolerance,

188
00:11:45,000 --> 00:11:47,790
maybe make it tighter
or a little looser.

189
00:11:47,790 --> 00:11:52,620
Now, if I wanted to allow
any value plus or minus,

190
00:11:52,620 --> 00:11:59,470
let's say, 0.1, I could
type this, abs equals 0.1.

191
00:11:59,470 --> 00:12:05,700
And this means that if convert
0.001 returns to me this number

192
00:12:05,700 --> 00:12:09,720
plus or minus 0.1,
I'll accept it as being

193
00:12:09,720 --> 00:12:12,580
equal to this particular number here.

194
00:12:12,580 --> 00:12:15,270
So a bit of tolerance
we've given ourselves here.

195
00:12:15,270 --> 00:12:17,190
I'll go ahead and run pytest.

196
00:12:17,190 --> 00:12:19,480
And we'll see that seems to have passed.

197
00:12:19,480 --> 00:12:21,900
But let's see how tight
we can make this tolerance

198
00:12:21,900 --> 00:12:24,040
before we get into some trouble.

199
00:12:24,040 --> 00:12:31,720
I might type something like this,
maybe 1e negative 5, let's say.

200
00:12:31,720 --> 00:12:34,985
So this is the same as writing
1 times 10 to the negative 5,

201
00:12:34,985 --> 00:12:37,740
a very, very small tolerance here.

202
00:12:37,740 --> 00:12:43,930
I'll go ahead and say pytest and
hm, we seem to have gotten an error.

203
00:12:43,930 --> 00:12:46,450
It looks like comparison failed.

204
00:12:46,450 --> 00:12:52,740
We see here that the actual
result obtained from convert 0.001

205
00:12:52,740 --> 00:12:57,900
was this number here, which is not
within the range of our expected value,

206
00:12:57,900 --> 00:13:02,190
plus or minus 1.0 times
10 to the negative 5.

207
00:13:02,190 --> 00:13:08,040
So if we loosen our tolerance a
little bit, I could do 1e negative 4.

208
00:13:08,040 --> 00:13:09,420
Pytest again.

209
00:13:09,420 --> 00:13:12,700
Even that doesn't seem to work
quite as well as we want it to.

210
00:13:12,700 --> 00:13:16,760
But if I do 1e negative
3, fingers crossed--

211
00:13:16,760 --> 00:13:18,010
even that doesn't seem to do--

212
00:13:18,010 --> 00:13:20,280
1e negative 2.

213
00:13:20,280 --> 00:13:22,630
That seems to be OK.

214
00:13:22,630 --> 00:13:26,340
So lesson learned here
is that you can actually

215
00:13:26,340 --> 00:13:31,630
adjust these tolerances to be
exactly the way you want them to be.

216
00:13:31,630 --> 00:13:35,310
And what I mean by that here is that
if I were actually willing to accept

217
00:13:35,310 --> 00:13:37,330
this tolerance, this would be great.

218
00:13:37,330 --> 00:13:39,990
But I might be writing some
scientific application,

219
00:13:39,990 --> 00:13:45,090
like maybe one for space exploration and
so on, in which case a tolerance of 1e

220
00:13:45,090 --> 00:13:47,560
negative 2 is really
not going to cut it.

221
00:13:47,560 --> 00:13:52,410
And so if I were to make this maybe 1e,
let's say, negative 12, for instance,

222
00:13:52,410 --> 00:13:58,320
a much more strict tolerance, and I
see that this test doesn't pass, well,

223
00:13:58,320 --> 00:14:02,430
I should probably do something in my
code to make sure that I get a value

224
00:14:02,430 --> 00:14:04,480
within this particular tolerance.

225
00:14:04,480 --> 00:14:06,900
So you probably shouldn't do
what I had just done here,

226
00:14:06,900 --> 00:14:09,910
which is adjust the tolerance
until you get the passing test.

227
00:14:09,910 --> 00:14:12,690
You should probably set your
tolerance first and then

228
00:14:12,690 --> 00:14:16,510
make sure your code is giving you the
right value you're hoping for here.

229
00:14:16,510 --> 00:14:18,370
In this case, I'll leave this alone.

230
00:14:18,370 --> 00:14:22,920
I'll say for now that a 1e negative 2
tolerance is sufficient enough for me

231
00:14:22,920 --> 00:14:23,470
here.

232
00:14:23,470 --> 00:14:26,760
But in the future, I want to
make this a little bit stricter.

233
00:14:26,760 --> 00:14:30,370
So here we've seen various
features of pytest.

234
00:14:30,370 --> 00:14:34,690
We've seen how to do simple
asserts and testing for equality.

235
00:14:34,690 --> 00:14:37,950
We've seen how, in this
case, we can test for raising

236
00:14:37,950 --> 00:14:40,060
certain errors, exceptions, and so on.

237
00:14:40,060 --> 00:14:43,350
And we've also seen how
we can test conversion,

238
00:14:43,350 --> 00:14:46,890
how we can actually go ahead and
compare floating point values using

239
00:14:46,890 --> 00:14:48,960
pytest.approx.

240
00:14:48,960 --> 00:14:51,220
This was our short on pytest.

241
00:14:51,220 --> 00:14:53,780
We'll see you next time.

242
00:14:53,780 --> 00:14:58,000