[MUSIC PLAYING] CARTER ZENKE: Well, hello one and all. And welcome back to CS50's introduction to programming with R. My name is Carter Zenke. And this is our lecture on testing programs. We'll see today all the ways our programs could go wrong, how to handle these things called errors, and see how to test our programs to ensure they behave as we intend. 

So let's jump in and see all the ways a function I've written could go a little bit wrong. I have here in RStudio a function I've defined called average. And this function average is defined in this file called average.R. And the purpose of this average function is to take as input a vector of numbers and return to me the single average number it finds across all of those numbers in that vector. 

So notice here how I'm using the built in functions sum and length. And you might know if you're familiar with averages or means that that's defined as basically taking the sum of numbers you have and dividing by the number of numbers you have. So it's exactly what I'm doing here with sum and length. 

And let me go ahead and presume that I figured that sum and length are correctly implemented. I can rely on these functions just as well in my own function called average. Now, it turns out there is already a function called mean, which does this very same thing built into R. It turns to us the mean or the average of some set of numbers. 

But our goal today is to write our own version of that function called average. So we can kind of see the design decisions that went into writing a function like mean in R. So here is my average function. Let's go ahead and try it and think about what could go wrong, actually. 

So I said before that this function average should take as input a vector of numbers. But we've seen some ways a user could give us not numbers, but text. If you recall using readline, you might know that readline by default takes as input text-- and hands it back text. So maybe I might have forgotten to convert that text to a number. 

I could run average in my console here, first defining it up above on line 1. I could run average. And let's say I've forgotten to convert some input from the user to a number. And I instead have now a vector of characters or character representations of these numbers here. So I'll pass as input this vector 1, 2, and 3. But those are not numbers, per se. They're actually characters here. 

I'll go ahead and run average. And now, I'll see this error. This is probably not the first time you've seen an error. Probably when you're programming, you've seen lots and lots of errors. But let's give these errors a more formal name. So these errors are more formally called exceptions. 

And an exception occurs when something exceptional happens in your program, but not in a good way. It happens when our program encounters some situation, some scenario it doesn't know how to handle. And instead, it stops entirely. So a question then becomes how could we handle these exceptions or these errors in our code? And one way to do so is to handle them more proactively. Preempt them. And do something else instead of encountering this error or this exception. 

So let's see if we could take that approach now in our own function here called average. I'll come back now to RStudio. And let's think through what exactly caused this exception. Well, if I look at it here, I'll see that I gave the sum function, it seems, some invalid type character of argument. So it seems like the problem was, in fact, that I gave as input to the average function this vector of characters. 

So what could I do to check for this before I maybe pass this input down into sum and length? I could probably use something like a conditional to ask some question. But what question would I ask? Well, I probably could ask is this vector numeric or is it not? And maybe I would consider the case where it isn't numeric, where I might get an exception to handle that case in particular. 

So here before I run line 3 now, trying to sum up these numbers, and finding their length, and dividing therein, why don't I go ahead and try to ask the question? Is, let's say, this vector x, is it not numeric? Just like this. So I'm going to make use of now this function, is.numeric, which asks the question, returns me true or false, is x a vector of numbers or is it not? 

And when I use this exclamation point here, I'm essentially asking is the vector x not full of numbers? And now I have the option here of handling that error before it might happen down here on line 5. So what could I do to handle this error? Well, a convention sometimes in the R world is to return a special value, one like NA. 

So if we give as input to our function average a vector that doesn't include numbers, I could say no, no, let's stop here and just return NA, instead of getting this error ultimately. So I'll go ahead and do just that on line 3. I'll say if we find that this vector x is not full of numbers, is not numeric, I'll go ahead and return NA instead. 

And hopefully I'll now avoid this error by kind of preempting it and handling it up above. Let me go ahead and redefine my average function now to update what it has included here. I'll go ahead and run the same thing I did before, giving as input to average this vector of characters. And I'll see I'll get back now just NA and no error. Now, we've handled it, preempted it before it has had the chance to arise in this case. 

But if we're going to do something a little unexpected here, like return NA when the user might have thought they were getting back a number, it's worth thinking about how to alert the user to that fact. Right now, we're handling this error silently, if you will. Meaning we're not going to raise anything to the user. We're going to hand them back NA. Unless they looked at the return value, well, they wouldn't know anything in particular was wrong or that they had done anything wrong. 

So let's think through how we could alert the user here and let them know what it is exactly we're doing here. Now, one way to do that is to make use of this function built into R called message. Message allows you to essentially send a message to the console while a function is running. So let's see if we could use message here. 

I'll go back now to my function. And maybe before I return NA, I could let the user know what it is I'm about to do. I could decide to send them a message using the message function. And it turns out that as input to this message function, I can provide the character string showing the message I want to tell the user. 

I want to tell them what it is I'm doing and probably tell them why I'm doing it. So first, I'll say that maybe this input x here, our vector, I'll say that x, this x here must be a numeric vector. So this is the cause of why I'm returning NA not, let's say, the actual average. And now, I could say what I'm doing instead. 

I'm going to return NA instead. Now, if I were to run this function, I need to first redefine it, go back down to my console, and provide the same input. And now, let's see what happens. I'll see that message. 

So now, we're not being silent anymore. The user who's run this function, they would get back NA as a return value. But now they would know it. They would say-- it would say x must be a numeric vector returning NA instead. So we've kind of gone away from being silent. And now, the user knows exactly what has gone wrong, perhaps, in this function. So a little more intuitive now. 

But it turns out that, by convention, message is often used when things are going just smoothly. We're trying to tell the user exactly what's going on. It kind of tells them a bit of a progress indicator. Gives them an idea of what their function is doing. It's meant to be used in cases where something has not gone wrong. 

But I'd argue, in this case, something has gone wrong. We gave as input to the average function an input it should not have been given. So there are ways to take a message and to escalate it, if you will, in severity. To let the user know that actually, something has gone wrong. There might be a potential issue here. 

Now, if we were to escalate this message, we could instead convert it into something called a warning. Now a warning is good when your function encounters something that is a potential issue. It's a bit similar to if you've driven a car and your check engine light pops up. Whether you're driving that car, you'll know that, well, your car could still continue running. But you might want to check under the hood and make sure everything is going as you expect it to. 

So a warning tells a user that something has gone wrong that could be a potential issue. And I think this is more in line with what's happened here. The user has given us a value that they really shouldn't have given us. So let's, instead of messaging them, warn them. And tell them that, look, you should not have done this. You should make sure this is exactly what you want in the end as far as the return value. 

So I'll convert message here to a warning instead, just like this. And now that means that the user will not get a regular old message. They'll get a warning indicating some potential issue. I'll go ahead and back to line 1. And I'll redefine this function. And now what will happen if I run it again with that same input, I'll see I get not a message, but a warning message. 

In this case, I see warning message in my function, and its input here, the exact message I typed on line 3, x must be a numeric vector, returning NA instead. So this is a way of alerting the user that they might get some value they didn't expect because there was a potential issue, which is they didn't give us an actual numeric vector. 

So a warning then is good for some potential issue in your function that it could still recover from. We could still return NA here. But there is one more level of severity, going from a warning to a full-fledged error. I think we could here have a discussion of whether a warning or an error is best for this scenario. 

On the one hand, I could argue that this function average is supposed to fundamentally take a vector of numbers and return to me a number. If I haven't done that, my function cannot accomplish its goal at all. In that case, I might not want to just warn the user and return NA, I might want to just stop entirely and say, look, you've given me an input. And I have no idea what to do with it. I can't handle it at all. I'm going to stop my function in its entirety. So let's see what it would look like if we actually not just warn the user, but stop the function entirely. 

Now, it just so just so turns out that R has this function called stop that allows us to raise or to throw this error. So let's upgrade now our warning to full-fledged error using stop, letting the user know that we simply cannot proceed with the input they have given us. I'll go back now to my average function. And let me go ahead and use stop, much like I used message and warning, I'll give it an error message in this case. 

But now what I should do is not return NA. In fact, if I were to run this code, I would never get to line 4 because as stop implies, my function will stop on line 3. It will not continue. It will not return any kind of value in this case. Why don't I go ahead and remove line 4 now? And now, what will happen is this. 

We're going to ask the question, is this input numeric? Or is it not? If it's not numeric, well, I will throw or raise this error that the user will now see. Let me go ahead and run or redefine this average function. Pass as input, the same thing we've been doing so far. And now, I'll see, well, an error. And we're kind of back where we started, giving an error now. 

But this one is more precise. It's one that we've raised or thrown ourselves. And it tells us exactly what has happened and why it has happened. Here we see error in our function average. x, the input, must be a numeric vector returning-- oops-- returning NA instead. And actually, that's probably not true anymore. So this stop on line 3 doesn't seem to return us anything, at least not NA now. 

So let's go ahead and go ahead and remove that message here to make sure that the user doesn't anticipate an NA. Why don't we just say x must be a numeric vector? I'll go ahead and redefine it. And now rerun it, and we should see exactly the error we were hoping to see here. 

So we've seen now how to talk to the user and message them about these kinds of potential issues in their functions. We've seen message. We've seen warning. We've seen stop. Let me ask now what questions we have about any of these functions so far, and how we convey or communicate about these errors that could happen in our functions. 

AUDIENCE: What's the difference between using message versus print versus cat to display an error message? 

CARTER ZENKE: So a good question. We've seen so far these functions like print, and cat, and now this one called message. They all seem to show us some text in the console. Well, a message is a more special kind of text output that we could later on choose to suppress. 

So you've probably seen so far, suppress warnings. There might also be a function called suppress message you could use to hide those messages as they come up. There is no such feature though for print row or for cat. A message is more particular and exclusive to showing the user a message they could either view or decline to view later on. Good question. 

OK, so let's consider other scenarios here that we could try to address in our function. We've considered so far what happens if we don't get the type of input we're expecting, in this case, a non-numeric input. But there are other scenarios we should probably consider and anticipate. And one of them might be if our input has NAs in it. 

So we've seen that the mean function, if it's given some input that has NA, well, it returns to us NA instead. So if we want our function to do the very same thing, maybe we could have a check here. Maybe after I check to see if the input is numeric, I could ask another question. I could ask this one here, if any, let's say, if any of the values in this x vector are NA, just like this, why don't we go ahead and do something else before we run, now, line 8? 

But what is it I should do if any of these numbers are NA? Well, I could, of course, return NA, like we decided to do earlier. But now there's a question here. I don't want to silently return NA. And I have three options. I could either message the user. I could warn them. Or I could throw an error using stop. 

Let me actually ask our audience here. What would you use? Would you use message, or warning, or stop in this case? How might you try to handle this particular input? 

AUDIENCE: I think it because is that-- if something or an error happened, it will-- the user [INAUDIBLE] the error. 

CARTER ZENKE: A good point. So we've seen that message is good for just conveying information, when there's nothing really wrong going on. Whereas, a warning is good to mention a potential issue the users should take a closer look at. And I'd argue that in this case, a warning is probably better. We're doing something a little unexpected. 

We're returning NA, as opposed to in this case, the average the user might have been expecting, so I think a warning would be best here to alert the user that there might be some potential issue here. I'll come back to RStudio. And let's go ahead and actually implement now this warning. I'll do it the same way I did before with warning here before I return NA. 

I'll give a warning. And in this case, I'll say as follows, that x, the input now, contains one or more NA values, just like this. So now, my function is looking a little bit better at handling these kinds of cases I might not have anticipated. First, if we get a non-numeric input, we're going to go ahead and stop. We fundamentally cannot continue with a non-numeric input. 

Then we're going to ask the question, if any of the values in our vector x are NA, we're going to warn the user and tell them that x contains one or more NA values. They might not know that yet. And we're going to ourselves return NA by convention here. If though, none of these conditions are true, we're going to go down to the bottom here. 

And we're going to return the average, just as we would otherwise. So let me go ahead and run this definition for the average function. I'll go ahead and give it, let's say, some faulty input, like we saw before. I'll get the error, like we see. Why don't I try now giving it an input with some NAs I'll give it 1, the number, 2 the number, and 3, NA, let's see. 

Now, I get NA back and this warning message down below. Well, let's try a normal input average, and I'll give it 1, 2, and 3, all numbers. Here we'll see the average of those numbers, 2. So I think our function now is much better designed. We're able to handle these edge cases that the user might have given us. And now, we can alert them to exactly what we're going to do to handle those cases. 

Now, when we come back, we'll see how to actually test this program this function in particular and make sure it's behaving like we intend. We'll come back in five and see how to test programs like these. 

Well, we're back. And we've seen so far how to preempt a few potential errors in functions we've written. What's next is to actually test our code and make sure it behaves as we intend. And we'll do so by writing what we'll call unit tests. Now, a unit test is some code that we write ourselves to test some unit of our program. But what are those units? 

Well, functions-- or sorry-- programs are composed of individual units called functions. So unit tests now are code we can write to test individual functions inside of our programs. And we'll go ahead and write some unit tests of our own by now testing our average function. So let's go ahead and do just that. I'll go back to RStudio. 

And I will, by convention, to test this function average, create a new file called test-average.R. So I will go ahead down here. And I'll say, I want to create a new file called test-average.R. And I'll see that file was created for me. If I now go to my File Explorer over here, I can open up test-average. And I'll see a blank page, in which I can write my tests for this average function. 

Well again, by convention, what I'll do is write a function to test this code that I've written now in average.R, in particular, this function average. I can call it test_average, just like this. And I'll make sure it is a function. Doesn't take any inputs for now. But within this test here that I've written, this function I can use to test my code, I'll then provide one or more test cases. 

Now, a test case is some representative scenario our function might encounter. And we want to ask the question, did our function return the right value for this particular scenario? So if I want to ask a question, I could do that using a conditional, as we've seen. 

So maybe I'll ask the question here. If, let's say, average, let me call our function, and give it as input this vector we saw earlier, 1, 2, and 3, all numbers. If the return value of average, given this input, is equal to 2, well what should I say? I could probably say, in this case, that average, my function average passed the test. And I'll give it a little smiley face, just for fun. 

Otherwise, though, if I don't get that value back, what should I show to the user or to myself here, the programmer? I could probably say something like, well, average failed the test. And give me a little sad face, to make sure I know what's going on. So this is my first test for this function average. Notice how I've defined one test case. My test case is when I give average the input 1, 2, and 3 as a list, I then expect that I'll get back the value 2. 

And if I do, I'll say the average passed the test. If not, I'll go ahead and say we failed the test. And now, for cleanliness here, let me go ahead and say backslash n to add a new line to each of these messages here. And why don't we go ahead and try to run this test_average function? 

Well, before we do so, we probably want to know a few things. One is I've only defined the test_average function here. I've defined it as having this test case here. But if I want to run it, I should still call this function, probably down at the bottom of my file down here. I'll say let's run test_average. 

And one thing you might notice if you're being particularly observant here is that I'm calling the average function. But at least within this file here, test-average.R, well I don't see average defined. What we don't want to do is this, I don't want to go over to average.R, copy and paste this, and put it over in test-average.R. 

What I can do more simply is run source within this file. I could say source, and then the name of the file I want to run before I run the rest of the code now in this program here. I'm going to run now the code in average.R, which will give me access to this average function. And I can then later on call it in this file. 

So we're kind of, if you will, importing this function into this file here. We're not-- we're now able to use any function we've defined in average.R in this new file, test-average.R because we've sourced it. We've run it before we've run any code in this file here. 

So I think this is all we'll need to test our average function. Why don't I go ahead and run this test here? I'll go ahead and click on source to run this file now. And we'll see, average passed the test. So it seems like in this case, if I give my average function the input 1, 2, and 3, it will return to me the value 2. 

Well, what are some other test cases we could think of? Like ideally, we'd think through some representative cases that we should know how to handle, but also ones that kind of cover a broad range of scenarios. Here, I've been testing positive numbers. 

But it would be worthwhile to test if average can work with negative numbers too. So let's add a new test case, I might just kind of copy and paste this for now. I'll take my test case here, and add a new one down below. And why don't I change now the input to the average function? I'll give it now some negative numbers, more representative examples here, negative 1, negative 2, negative 2, and negative 3. And what should I get back? Well, negative 2 is the average of negative 1, negative 2, and negative 3. 

That's what I should expect. Here, I've tested now both positive and negative numbers. But it's probably worth testing zero too, which is neither positive nor negative. I'm trying to think of scenarios that might go beyond the usual cases but are still important for me to be able to handle appropriately. So why don't I make a new test case? One that involves zero? 

I'll go down here and add a new one. Maybe I'll do negative 1 and 0 to use that number and then 1. So now we're going between negative and positive, and neither negative or positive. And the average here should be, well, zero. So here, I have three test cases in this one test function. First, I'll test positive numbers. Then I'll test negative numbers. Then I'll test positive, negative, and neither positive nor negative numbers, hoping for the right output in each case. 

I'll still run my test average function down below. Let me clear my console, click on source. And now, I'll see average seems to have passed all three tests. So my code seems doing pretty well here. But if we wanted to keep going and adding more test cases to this, I'd argue that we'd get pretty bored pretty quickly. And it would be a lot a lot of copy/paste. Like I've already written here 21 lines of code to test a function that was 10 lines of code. 

And if we wanted to test our programs, and every time, had to write three, four times the amount of code to test that function, well, nobody would test their code. And we want people to test their code. So thankfully, people who are in the R community have developed their own package to allow us to make testing easier, and arguably, more fun. 

So a package that is canonical in the R community to test your programs, is called testthat. It allows you to test that your function behaves as you might expect. So let's go ahead and use testthat now to improve the design of our tests and make it easier to write test cases like these. Now, testthat comes with a function called test_that, which allows me to make a new test for my code. 

But before I can use it, I, of course, need to install testthat. So if you haven't already, let me go down to your console down here and say install.package, and install testthat. Once you've installed it, you then need to load it. So I'll go ahead and load testthat, just like this, by doing library followed by testthat. 

Now I'll go ahead and Enter here. And I'll see that I've now loaded testthat. I now have access to functions like test_that. So I think what I've written here so far is pretty good. It at least has some good test cases. But I don't need any of this to use test that anymore. I'm going to go ahead and delete most of it, but still include now my average.R import, if you will. 

I'm taking whatever I've written an average.R and making it able to make me-- making myself able to use it now and test-average.R. Now, we said before that testthat comes with a function called test_that. And we use this function to define a new test for some function that we have. So I'll go ahead and go back to test-average.R. 

And I'll go ahead and use test_that. And the first input to testthat is a description of the test I want to run. So here I'll say I want to test that-- I want to test that-- oops-- that average, let's say, the average function here, calculates the mean, or in this case, the average of these numbers. 

So this is kind of an English sentence now. I'm going to test that average calculates mean. Well, the next argument is the set of test cases I want to run to ensure that testthat-- or to ensure that average calculates the mean appropriately. By convention, I'll put these test cases inside of these curly braces as a second argument now. And I can now provide several test cases inside of this one function that I've decided to create here. 

Now, how could I say-- or express a test case? Well, testthat comes with some functions we can use. And we really use them by expecting-- or saying what we expect to happen when our function returns some value. One of these functions here is expect_equal. We could expect that when our function is run, we should get back a return value that is equal to some other value, much like we just did with our conditionals earlier. 

But now, I'll use expect_equal. Let me go back now to my code. And inside of my test here, I'll go ahead and define a few test cases. The first one will be I want to expect equality between the return value of the average function when given 1, 2, and 3 as input, and this value 2 on the right hand side. 

So to be clear here, the first input to expect_equal is the argument, the value we'll get back from, in this case, our average function. And the next argument is the value we expect to find as the return value of average. I'm going to expect those are now equal. 

And this is our test case. There are no conditionals, no nothing else. We're going to go ahead and just use this to test, did average return to us 2 when we gave it as input a vector of 1, 2, and 3? Well, let's now add our other test cases. I could copy/paste this and change the input. I'll do negative 1, negative 2, and negative 3 to test now for negative values. The expected value is negative 2 now. 

I'll do the same now. But for negative 1, 0, and 1. The expected value now is going to be zero. And why don't we go ahead and just add some more test cases? Now it's just so easy for us. One thing I could do is test maybe more than an odd number of numbers. I've always been testing three here. Maybe I'll test four as another scenario. I'll go ahead and do, let's say, negative 2, negative 1, 1, and 2. 

So now we test both positive and negative numbers. But now, we're giving an even number of numbers as input. And we should get back, of course, zero in the end. So this, then, is our test of our average function. Let's go ahead and see what could happen here. Notice how at the top of RStudio, I now see a button called Run Tests. 

This means we're going to run every test we see in this file. I could, alternatively though, go down to the bottom of my console and just source this file to run it. I could say source test-average.R and let's see what kind of output we get. I'll go ahead and run this. And oh, test passed. We have a little gold medal here to say our function worked as we intended it to. 

Here, I can see that average will return to me all these values for each of these test cases, making it much easier now to write test cases like these, thanks to testthat. Let me ask now, what questions do we have on defining these test cases and using a package like testthat? 

AUDIENCE: Don't we have to source average data because if you have a huge file, and if you only want to test one function, then it won't be like a good idea to source the entire file? 

CARTER ZENKE: A good question. So notice here how I actually ran the source test-average.R because my goal was to run these tests here top to bottom. Test-average.R already sources or runs, if you will, average.R, giving me access to any functions inside of that file here. 

When we come back another time, next lecture, we'll see how to make packages of our code. And we'll see how to write tests that don't require us to put source up top. But so long as we're not running packages and just testing our code, we're going to need to include source average.R up top to give us access to average. So we can run it inside this test file here. What other questions do we have on testthat or testing our code so far? 

AUDIENCE: When is an appropriate time to write tests? 

CARTER ZENKE: Yeah, when is it appropriate to write tests? And what time is appropriate to write tests? So there are-- so I'd say any varying philosophies on this. There is a kind of a movement or a philosophy called test-driven development, which argues you should write tests before you even write your code. 

And by writing your tests, you kind of get your mind around what you want your code to do. And then you write code to pass those tests. On the other hand, folks might say, well, I just want to get something done, I'll write the code, and then I'll test it. There's arguments on both sides to be made. It's going up to you and your team to decide when you want to test and how you want to test. 

This is telling us how we could test now using packages like testthat. But good question on when to test as well. OK, so here, we've written a pretty good test case. There are many test cases here for average function. But there are still other scenarios to test. And in particular, we saw what could happen if we gave average some input that included NA values. 

Well, we could just as well test the result of average when it's given some NA values as we could some regular values like these. So let's go back now and add some new tests and test cases to our file here. Now, if I want to test what average does, when it's given some input that includes NA values, well, I could keep adding test cases here to my single function, or my single test, average calculates mean. 

But if I were to keep going and adding more and more tests, this function would become quite, quite long. So ideally, what I want to do instead is maybe divide up my test, my test cases, into a way that makes logical sense. 

Here, I argue, I'm going to have all my test cases that are giving average some pretty typical inputs, numbers, I'm going to find the average of them, and get back and check for equality. But if I'm going to give average some new type of input, like inputs that include NAs, well, maybe I should make a new test for that. And I can do that by including more than one instance of this testthat function. 

I could say I want to test that. Now, average, let's say, how do I want to word this? I want to say test_that average warns about NAs in input. So we saw before that our goal, when we wrote the average function, was to test and to make sure that it gave us a warning when the input x included NA values. 

So we could write some test cases to make sure that that is what is happening with our average function. So here's my description of this test. I'll go ahead and give myself some space now for test cases. And I want to test that average raises or throws a warning here. And it seems like expecting equality might not work because this allows me to test two distinct values. But a warning is something else entirely. 

Well, thankfully, in testthat, we have access to other expectations we can say, one including test warning, or expect_warning in this case. We can say we want to expect a warning from this function or expect no warning at all. 

So let me go over here and say I want to expect that I'll get a warning from the average function when I give it some input like this, maybe 1, NA, and 3. So some input that involves an NA. I can do the same thing down below. And I could say, why don't I give it maybe all NAs here, a vector of three NAs? And now I could expect that when I run average in this way, I expect I'll get a warning. 

So let's go ahead and rerun our tests now, testing for both a calculation of the mean and a warning from average. Let me go ahead and run this. And oh, what do we see? Test failed. So let's see what happened here. If I scroll back up, I'll see that one test passed. That seems to be my first one here, average still seems to calculate the mean. 

But if I look down below, my next test is that average warns about NAs in the input. And in fact, what I've gotten it seems, from average, is not a warning, but an error, error in average. When I gave it NA, NA, NA, x must be a numeric vector. So although I expected a warning on line 12, it seems like I got an error instead. 

So that's probably cause for me to go back to my code, and see what could happen, so I could fix it, and make sure it adheres to these expectations of my function. Let me come back now to average.R and think through what could be going wrong here. Well, we got, it seems, this error, that x must be a numeric vector, when we wanted, it seems, this warning down below. 

So maybe what happened is that when I gave it a vector of all NAs, maybe it found that that vector is not numeric, which it might well have done. So let me go ahead and get on my console here and test this. I could say is.numeric, and give as input NA, NA, NA, and ask is that numeric or not? 

Hmm, so it's not. So because this vector of NAs is not numeric, I would first throw my error that x must be a numeric vector. But what I really want to do is return NA if I get a vector of NAs. So I think I should probably reorder here this handling of my errors. Let me go ahead and reorder this and put this up top first. 

So now, what we'll do is first check. Is NA-- or is the vector-- is the vector we got as input to average here, does it include any values? If so, we'll raise a warning and return an NA. And then we'll check if it's numeric. I think this might help us solve our problem here. Let me go back to test-average.R. Let me rerun these tests with our updated version of average.R. 

And we'll see all the tests passed. We have a little confetti, some rainbows. We seem to be moving along pretty well. So what questions do we have on this new version of our test? We've expected how average will calculate the mean. And will we get a warning, now? What should we do next? And what questions do we have before we move on? 

AUDIENCE: When you get-- when the user gets a warning, can we use pass to get the user to rerun the input to have a warning in-- or an error? 

CARTER ZENKE: So I'm hearing a question about handling these errors or warnings as they come up in our code. And it turns out that R actually has a function called try and a function called try_catch that let us handle errors and warnings as they arise in our code. We won't focus on those today, but certainly learn more about them if you're curious about them too. 

Let's keep going here and see what else we should test. So I argue that we've tested that average returns us the right value to calculate the mean, that it also warns about NAs in input. But what else could we test? Well, it seems like we should also test that average actually returns to us NA if we give it NA value. 

Here, we're only expecting a warning. And we're not so much testing if we're getting back the right return value. So let me do just that. I'll go ahead and add a new test case, one that tests that average returns NA. I'll test that here. Average returns NA with NAs in our input, just like this. 

And I'll go ahead and add some more test cases. Well here, it seems like expect_equal might work for me. I'm going to test that the return value of average will be equal to an NA value. So I'll go ahead and expect_equal. I'll go ahead and give the same kind of input I gave down below here, 1, NA, and 3. I'll expect that to be equal now to NA. 

Let me go down over here and say I expect_equal between this vector of just all NA values and the NA value itself. Let me go ahead and actually make this a vector, just like that. Make this a vector as well. And now we're passing into average those same inputs down below. But now, I'm testing to see if the return value is NA. I'll go back to my console now and run these tests that I've just added. 

And we'll see, hmm, something a little bit curious. I'll see test passed. And I'll see test passed. But I'll see a warning, it seems, in my second test. That average returns NA with NA as an input. And I get back a warning that x contains one or more NA values. 

Well, that is kind of expected because if we look at our average function, we'll see that if we do give our function NA values, we're going to throw this warning. But what we're testing in this test is not so much that we get a warning or not, we already did that down below. We're testing if the return value is equal to NA. 

So this would be a good chance for us to use a function like suppressWarnings. We're saying, we don't really care about the warning, we get we just want to test the return value in this case. So I'll wrap the average function, in this case, inside of suppressWarnings-- suppressWarning-- suppressWarnings, sorry, let me make it plural up above. 

And now, I think we should probably solve our problem here. I'll go ahead and rerun these tests. And now, we'll see, I have three tests passing overall. Well, what else could we test? We saw before in average.R that we also want to stop. We want to end our function, throw an error if we give it some non-numeric input. 

We could just as well test for that. I have this other set of functions, thanks to testthat. One is called expect_error and expect_no_error, those test if my function has stopped given some given input. So I could use now expect_error. We come back to test-average, and go down below and say, I'll test that maybe average stops if x, our input, is non-numeric, just like this. And now, I'll go ahead and expect that I'll get back some error if I give, as input to the average function, some value that is not numeric, maybe something like-- something like quack, just like this, or something like that test we saw before, 1 as a character, 2 as a character, and 3 as a character. 

So this is non-numeric input. Let's see if we get back the error from average. I'll go ahead and run. I'll first save my file. I'll then run test-average. And now, I'll see four tests passing too. So here, we've seen how to write test cases for our code thanks to expect_warning, expect_equal, and expect_error. What other questions do we have on testing our code using these kinds of test cases and testthat more generally? 

AUDIENCE: One thing which comes up very much in computer science is floating point inaccuracies, right? So can we account for that? 

CARTER ZENKE: A really good question. Actually, an excellent segue I was just going to talk about now. So it seems like in our code, we are testing integer numbers. I have here, 1, 2, and 3. And we get back a whole number, like 2. Same for all these other test cases. But to your point, we've missed an important kind of test case, which involves these floating point or decimal numbers. 

And due to what you've said about decimal numbers, and the way they're represented, there are some special considerations we take into account before we can test those kinds of numbers. So let's see what we should take into account before we test, in this case, floating point or decimal values. So let's actually go ahead over here and think through how we could test it, at least hypothetically. 

I could use here expect_equal still. And give now as input the average function. But I'll give it some floating point values. Maybe in this case, I will try to test these, 0.1 and 0.5, taking the average of this, which will be 0.3. 

Now, it seems to us, as humans, that if we were to do this kind of calculation, we would get the following kind of answer. That 0.1 plus 0.5 divided by 2 is equal to exactly 0.3. This is the average of these numbers, 0.1 and 0.5. 

But it turns out that computers can't do math exactly like this. That there are actually an infinite number of floating point numbers, of decimal numbers, and only a finite number of bits we can use to represent them. Which leads to this problem known as floating-point imprecision. We have so many decimal numbers to represent and only so few bits represent them that we can't represent all of them precisely. 

And in fact, a computer, even one running R, might perform that same calculation and arrive at this, 0.1 plus 0.5 divided by 2 is equal to 0.299999-- lots of nines, then a lot of other values after that. So not exactly 0.3. 

And so because of this, we need to allow for some tolerance when we test our floating point values. But before we do that, let me kind of prove to you that this is what's happening in R, even. I'll come back to my computer here. And let's get an idea of this imprecision that happens in floating point values. 

Let's go back down to my console here. And if I were to print, let's say, this value 0.3, I'll get back 0.3. But if I push R just a little bit and I ask it what really is 0.3? I could say print 0.3 and show me now 17 digits after the decimal point. Let's see what we get. 0.29999999999-- so it turns out that 0.3 is just one of those decimal numbers we can't properly represent due to having so many floating point values and so few bits to represent them. 

Now, because of this reality, like we said, we do need to allow for something called tolerance when we're testing code like this. Now, tolerance is the range of values I will accept above or below my expected value as being equal to that expected value. 

Mathematically, we could say this, maybe my expected value is 0.3, but in terms of numbers being equal to 0.3, I'll say any number between this range here, plus or minus let's say 1 times 10 to the negative 8, some small number here. Now, this number is our tolerance. 

And we can change it, if we wanted to too, depending on our needs for precision, or how we're presenting these numbers here. I'm gonna come back now to RStudio and show you that expect_equal actually has a parameter called tolerance that we could use to change this value here. 

I'll come back over. And let's look now at expect_equal. If I clear my console and go here, and say, in this particular test, I want to set some tolerance here. I could say as another parameter, tolerance, and set it equal to some small value, let's say 1 times 10 to the negative 8, which I represent now as with 1e negative 8 here. 

Now, it turns out that testthat gives you some fault tolerance that's already been decided. So I'm actually going to rely on them to choose my tolerance for me here. But you could, if you wanted to, override it with this parameter called tolerance. 

So we've seen now this idea of floating-point imprecision, and this idea of tolerance, which we can use to better test our floating point values inside of our tests. Let me ask now what questions we have on either of these topics here. 

AUDIENCE: Why writing code spending much time to test another code, while you can go to the code that you wrote and just test it simply? 

CARTER ZENKE: A really good question. So maybe you're in the habit of kind of just testing your code in the console, for instance, kind of like I did earlier. If I come back over here, maybe I wrote the average function and I decided I'm pretty confident this will work. I'll assign it to be the average here. And I'll just run a few test cases, like average c, 1, 2, 3 here. 

OK, that seems to work. Maybe I'll do average, and then maybe c negative 1, negative 2, negative 3. That seems to work as well. Now, the reason you might not do this and instead spend more time to write your own test is simply just robustness of your tests. 

Notice, I can very quickly test for a lot of different scenarios in my tests here and make sure that my function works as it expects. It's also very useful if you're collaborating with others. Let's say you're both-- somebody else and you are both writing the average function. Well, you could collectively decide on what tests you will run to make sure that the average function is correct. 

And if somebody later were to go into the average function and make a change, they could test to make sure that their change did not break the code altogether. So this is a good way to standardize what it means for your code to be correct as well. But a good question on why would I even spend time writing tests like these. 

OK, so I think we've so far seen a lot of good tests for our code now. When we come back, we'll see how to test not just numbers, like we did here, but also strings as well and focus on these two philosophies, one called test-driven development and one called behavior-driven development. We'll see you all in a few. 

Well, we're back. And so we're going to next focus on testing these return values that will be strings, as well as focus on a few philosophies of testing, namely test-driven development and behavior-driven development. Now, what is test-driven development? Well, it is an answer to when and how we should write our tests. 

And central to this philosophy is that tests should be at the heart of your development process. In fact, it even argues you should probably be writing tests before you write the code. Now, let's consider here, I want to write a function that says hello to a user, one like a greet. 

Well, to make that happen, I should probably first write the tests for that code and then write the code itself. So let's do just that now over in RStudio. Come back over here, and create a file that will test this function called greet that doesn't yet exist. 

I'll do file.create. And then I'll say I want to create this file called test-greet.R. And I'll say-- it was created here-- I'll go ahead and go to my File Explorer and open up test-greet.R. And now I could start defining some tests to test this code that doesn't even exist yet. But why would I do that? 

Well, by writing tests, I make it much clearer to me and to others what it is I want this code to do. And once I have in mind what I want the code to do, I'm better able to write that code itself. So the very first thing I probably want to test here is what that test-- the greet function that it can say, let's say, hello to a user, just like this. 

That's the core part of this greet function I'm going to write, that it says hello to a user. And now, I could define some test cases to make sure that this is the reality. Why don't I go ahead and define at least one? And I'll say I'm going to expect that if I were to run greet and give it as input, Carter, I would get back as the return value now hello, comma, space, Carter. 

So here is my very first test and test case for this function greet that doesn't yet exist. I'm going to say that I want to be able to use greet in a way that it passes in a user's name and returns to me then the user's name, but with a prefix of hello, comma, space. So that is our very first test. 

Let me go ahead and make sure I include this eventual file that I will create called greet.R, in which I'll define greet itself. And once I've done this, well, I could probably start developing. Let me go ahead and now go back to my File Explorer. 

And I could either create a new file by hitting this plus button here. Or I could go ahead and do file.create. I'll use greet.R, hello.R. And then I'll go ahead and go to File Explorer here and open up greet, this blank canvas for me here. And now, I could define my greet function to do exactly what I see it should do in my test. 

Well, I'll say I have this function here called greet. And I'll define it as a function that takes some input. Maybe in this case, the input is called to-- because we're going to say hello to someone in this case. I'll go ahead and say that this function should return some value. And we've seen this function called paste before that concatenates strings. I bet that's what we might need here. 

I'll use paste just like this. And I'll paste together hello, comma, and then whatever value is supplied as input to this function under the argument or parameter to. And notice how paste here will take care of the space between the comma and whoever we're saying hello to. 

So now, thanks to my test, I have a very clear idea of what this code should do. And if I were to run this now, test-greet.R, we'll see that the test passed. So it seems like now, greet is working for me. I could go ahead and add more test cases. In fact, this is an iterative process. 

I might write some tests, write some code, write some tests, write some code. Here now, I could test other names. Maybe I'll say expect_equal, I'll greet Mario, and hope to see hello, Mario. I'll do maybe Peach as well. And hello to Peach. I'm just choosing some representative names I might get now and pass into this greet function. Let's do Bowser as well. 

So now, with these expanded test cases, I'll go ahead and test my code again. And I'll see that it still seems to be working. And now, if I were to modify greet in my greet.R file, I could very quickly test to make sure I didn't break it with any changes that I had made. 

So this is one philosophy of development, test-driven development. But there is a related philosophy that is still interesting to learn about, one called behavior-driven development. So test-driven development focuses on designing these test cases for our code, giving it representative cases and seeing if it actually follows through on the expected values. 

Behavior-driven development is slightly different in that it requires us to first define what it is we want our function to do and describe its behavior. Now, testthat allows us to use behavior-driven development and actually gives us a few functions we can use that kind of operate a bit like an English language to define what it is our function behavior should be. 

Let me show you what I mean. So I'll come back over here. And the two functions we have in testthat to engage in behavior-driven development are these, describe and it. Where describe is a way of describing what it is we want our function to do. And it is a way of saying that our function should do something in particular. 

So let's go ahead and switch now to this philosophy of testing. I'll avoid using testthat. And now, I'll start using describe. So in particular, describe lets me do something like this. I want to describe now my greet function. And by convention, stylistically, I'll include these empty curly braces here. 

And now I can say inside these curly braces, what it is I want my function to do. Inside these curly braces, I'll describe what my function now should do using this it function. I could say that it, in this case, can say hello to a user. And as a second argument now to it, I'll provide some test cases that show examples of it saying hello to the user, namely this. 

I could say maybe I have this object called name, I'll set it equal to Carter. And I'll expect that when I run greet and pass as input name, I should see hello, Carter. Similar now to before. But notice what it is we've done. We've kind of used a bit of English language in the form of these functions. 

Here, I'm going to describe my greet function. Well, what should it do? It can say hello to a user. And here's an example of it doing just that. So pretty cool. Let me go ahead and now run test-greet, using this other philosophy here, test-greet.R. And I'll see that test has passed. 

What else could our function do? What kind of behavior do we want it to exhibit? Well, maybe I could also say that it, describing greet now, can say hello to the world, just like this. And now, provide an example of it saying hello to the world. I'll expect equal that when I run greet, without any input at all, I'll say hello to the world. I get that back-- I'll get that back out as a return value now. 

So here, we see a fuller version of a description of greet. First, it can say hello to a user. And it can say hello to the world. And by convention, I'm using this can say, can say here because when we use it, it reads more like English to say it can do this, it can do that. So I can mention here, we're just using these-- that grammar here. But I could use any kind of text inside this it function here. 

Let me go ahead and try to run this. I'll say source test-greet.R, and we get-- oop-- seems like, if I scroll up now, that one of our tests has failed. So here we see error in greet. Greet can say hello to the world. But we actually get back an error, and not, in this case, hello, world. Error in greet argument to is missing with no default. 

So let's look now at greet.R and see what could have happened here. I ran greet with no input. And if I go look at greet itself, well, now it kind of makes sense because I didn't supply a default value for to if none is supplied. 

So what should I do? Maybe supply a default value here? I could go ahead and say that to has a default value of world. And fix this code after I have described it how-- described how it should work already. Let me go ahead and now rerun these tests with this updated version of greet. And we'll see that both of my tests have passed. 

That greet can say hello to a user. And it can say hello to the world. So we've seen now how to test these strings for equality, how to use test-driven development and behavior-driven development. So what questions do we have on test-driven development or behavior-driven development? 

Seeing none, so let's focus on last-- one last topic for today, one called test coverage. Now, the goal of today has been to help you write tests that systematically test your code. And one measure you can use to figure out how much of the code you're testing is one called test coverage. When you have programs that are composed of not just one function or two but many, test coverage and tell you how many of those functions you've tested reliably. 

Now, we've seen today how to spot errors in our programs, how to handle those errors, and how to write tests to test our code to ensure it behaves as we intend. When we come back, we'll go ahead and see how we can package our code up and share it with the world. We'll see you then.