CARTER ZENKE: All right, and for this session, we'll transition from you being students to you being teachers of CS50. And as a teacher, one of your goals is to give feedback, to tell students what they're doing well and what they could be doing better. That's a really important skill. And so today, we're going to practice doing just that. How do we give good feedback for people who are learning computer science? How do you help them succeed in what they want to do in computer science? So I have a few goals for today. The first one is to learn about good design of code. I assume that when you submitted your problem sets, the code was correct. Yes? Sometimes, at least, the code is correct sometimes. It worked. But there's another question. How well did it work? Could it have worked better? Was it efficient, or could it have been more efficient? So we'll learn today about code design. How well is this code written? How could it be improved, even if it is correct? We'll also practice grading code, practice giving feedback. You'll take a look at some examples of student code that real students have written, not you, but real students have written. And you'll practice giving feedback on that code as if you were their teacher, OK? How could you help them design a better program? Our next goal is to learn to differentiate between better and worse design. What makes one program better designed than some other program? How do we know what to look for to see is this code well-designed or is it not well-designed? These are our three goals for today. YULIIA ZHUKOVETS: All right, so next, we're going to transition to talk about three grading axes. So CS50, across Harvard College, Yale College Extension School, the X community and now for Indonesia teachers, really great along the three grading axes. The first of them is correctness. So something all of you already familiar with correctness is when you get all smiley faces on check50, right? So when your code runs well, when it executes according to the spec, when it passes all the tests. So one of the grading axes is correctness, using the check50, the algorithm, right, that Rongxin and the team built back at Harvard. And this is quite simple, right? You just click a button or run check50, and then it spits it out in the terminal if the code was correct. The next grading axis is style. So some of you might be familiar with was a little tiny style50 button on top of the CS50.dev environment. If you click that button, it will tell you if your code is indented correctly, if there are enough spaces. Maybe you have too many spaces, right? It would kind of tell you how you can make your code more aesthetically pleasing, right, so that it's very uniform and it's looking according to some standards in computer science. Now, there is the third axis that you might not be familiar with which is called design. And so this is exactly what we'll be talking about today. And what exactly is design, right? We talked about code being correct. We talked about code being aesthetically pleasing. Design is how Carter earlier mentioned is really about having efficient code, having that one perfect solution, where you don't have five different FOR loops and 10 different IF statements, right? It's a very succinct, so simple code that someone else that's not maybe familiar with CS50 can easily trace and understand what you, as a programmer, wrote. Code that has good comments and variable names. So all of those things are called design. And unlike style50 or check50, this is not exactly something that we can program computer to do, right? Like, it can maybe check for the number of comments, but it can't really check if the comments are relevant, right? Maybe I'm writing completely random comments that are not related to my code. Well, that's bad design, OK? So today, we will actually talk about different programs. So we'll jump into readability, Caesar, and then you'll get to then grade and give feedback on some other problem set that I'm going to keep in secret for now. It will be a little surprise for you. But next we're going to jump into actually how to grade design, right? I gave you some overview to think about maybe inefficiencies or how to make code more succinct and simple. But Carter will actually help us understand how to grade it, how to assign it a specific score. CARTER ZENKE: As Yuliia mentioned, the idea of design is that we have to grade this as humans. A computer can't tell us whether the code works or not. We have to actually say is this code as well designed as it could be? It requires your input as a human, as a teacher, to talk to the student and say, how could you improve what you wrote, OK? So the code is good, but how could you make it even better, right? And at CS50, we have one suggestion, which you can take or not, which is to grade design on a five-point scale. Five is the best. It's the perfect design. This code could not be any better written. Cannot get better than that. Now, below a five would be a four, which is still great design. The student did well. They used a lot of the concepts, but there are still one or two potential improvements, things they could have done to have been even better with how they wrote their code. Next would be a three, which is good design. The student did good, but there were still one or two inefficiencies that they could have improved, things they could have improved to make their code work better, right? Below a three would be a two, and below a two would be a one, if we can go ahead. And this symbolizes kind of worse and even worse design. That the student solved the problem correctly, but they didn't do it in a very efficient way. There are things they could still improve, very much so, in fact. So we think of these things as like a one to five scale, where five is the best design, and one is the worst design, OK? So this is only one suggestion. You can take your own ideas, too, and apply them here. Sometimes I will like to use a three-point scale, where three is the best, and one is the worst. You can even do a 10-point scale, from 1 to 10. You could even not use numbers at all. You could use words, like excellent or great or good or could be better, could be improved. It's up to you. But for today, we'll use CS50's approach, which is five-point scale. Five is the best. One is the worst, OK? And most students in CS50, like the code you'll see today, might not be a five. Five is perfect. Cannot be improved. Most students submit code that is usually a three or a four. It's good or it's great, but there's still a few things that they could improve, OK? All right, now let's see some examples. YULIIA ZHUKOVETS: All right, so we talked about correctness and style and design and how to grade it, perfect code or code that can be improved. Now let's actually see some examples so that we can practice giving the score. And maybe we can also even do a little vote ourselves. So our first example would be on readability, right? The P set that everyone should be familiar with, where the goal of the P set was given a prompt from a user, which is some text, calculate the number of letters, words, and sentences, and then using a specific formula, we can calculate an index that will tell us maybe how complex that text is. So today we will take a look at two snippets of code, where we specifically focus on calculating the number of letters. So before we jump in, let me just take a few minutes to walk through the code to explain exactly what's happening. So first we can see that we have a main function, right? There was some code that was written before. Maybe we did get string to get user's input, and we processed that. And so maybe we already calculated some words and sentences, but this is specifically a part for calculating number of letters. So first thing that we see is declaring a variable, called L. And we're giving it a type float. We're setting it to zero. This will be sort of our counter that we will use to count the letters. Next, it seems like the student that wrote that code used a FOR loop to iterate through all of the letters in the text, going one by one. Sorry. The characters in the text, going one by one. And next, it seems like they implemented an IF statement to check each character, if each character is between a certain bounds of numbers. Maybe I'm not familiar with CS50 and I'm just looking at this code. These numbers don't really mean anything to me just yet, but I can see that in the next line, they added one to our variable L with every iteration, and there is some code on the bottom, right? Maybe this is where we implement our index formula. And then Carter will talk us through the second example. CARTER ZENKE: So you've seen one example that a student has submitted to us, as teachers, but you probably have more than one student. And so this is our second student, now, who submitted another assignment for the same problem. Still readability but this was the code that they wrote. Now, it's important, if a student submits code to you, you should try to first understand it. Can I tell you a secret? Yeah? Can I tell you a secret? OK. My secret is sometimes a student will send me code, and I don't understand it. [LAUGHTER] That might happen to you. OK? And that's OK. But what you can do is walk step-by-step to try to understand the code they wrote, OK? So we'll do that today. Looks like here, at the top, we have function main. What does main do? It's the start of our program, yeah. So the code starts up top, and it seems like with a dot dot dot-- do you see that, slash slash dot dot dot? Do you see that? Yes? That means there was some code up above that they wrote. We aren't going to show it here because it would be very long, too big for this slide, OK? But then we see int letters=count_letters text. So what does that mean? It seems like that the text we got from the user, we're giving that to our function, which is called what? Count letters. Good. And what does it give us back? It gives us back some number of letters, I think, because here we have a variable named letters that is an integer. So we'll get back a whole number, like a one, two, three, four number of letters in the text that we had, OK? OK, down below we see int count_letters string text. Do you see that? OK, good. That's the start of the function here. Then we see a comment, oh, a comment that makes things so much easier. So ask your students to write comments, right? You can help them understand their code. This says it returns the number of letters in the text. That's pretty easy to understand. Yeah, that's helpful, isn't it? Good. So we know that this code here from int letters=-0, all the way down to return letters, its goal is to return the number of letters in the text. OK, what's the first thing we do? Can I ask you this line here, int letters=0. What does that do? Yeah, assigns the value zero to this variable, named letters. And what type is it? Integer. Good, good. OK. So we have a number, zero now. Its name is letters. So it seems like we'll count up the number of letters using this variable named letters. Does that make sense? OK, now we see a FOR loop. Uh-oh, a FOR loop. So what does the FOR loop do, do you think? If we read just this part. FOR int I=0 len=strlen I less than len, I plus. Plus. Ooh. What does that do? I'm hearing some good ideas. So I think it iterates-- remember that word iterates? It iterates over every letter, every character in the text. So if I had maybe "hello," that's five letters, yeah? So I would first get H, then E-L-L-O. Good. OK, then I would ask a question for every character. Like, H-E-L-L-O if is alpha, uh-oh. If it's an alphabetical letter. So alphabetical letter means A through Z, right? OK, so if it is A through Z, this particular character, what will I do? Add one, OK? So I'll add one, and once I get to the end of my text, it seems like I should have counted up all the letters. So for instance H, H is certainly between A and Z, yes? Add one, so from zero to one. E, add one again. That's two. L, add one again. That's three. Add one. Add one. That's five, right? So we counted up all the letters in this text. So we've seen now what this code does. I think it's correct. But now we could compare these two students' design. Which one was better written? Which one was easier to understand? And which one was more efficient? So here, again, are these two pieces of code side by side, and let's think through which one might be better. We'll demonstrate this first, me and Yuliia, and we'll later have you all do this same process in your groups, OK? So we'll demonstrate now. But you'll prepare yourself to do this later on for more code examples. So let me ask you, Yuliia, what do you think about the design of these programs? YULIIA ZHUKOVETS: Well, as you said, the functions seem to be correct. I think they're calculating the number of letters in the text, but I think there's some things that jump out to me that could be better. Maybe on the left-hand side, for example, I can notice that the example 2 on the right-hand side uses a helper function count letters. So inside the main, I really only need to write one line, where I will assign the return value to some variable letters. And then all the way on the bottom, I can create a separate helper function that will count those letters. It seems to me like that's better design, because it doesn't crowd my main function, and I can really focus on implementing the most fundamental steps, like getting the user input and calculating the index and kind of like abstracting away everything else. And kind of like comparing it to the left-hand side, I see that there is no helper function everything is implemented in main and given the amount of code it might seem to me like the example 1 main function can get really long. CARTER ZENKE: Yeah, I agree with you. I like that this student made their own function called count letters, which seems to me to be a little bit clearer, maybe a little bit more efficient, let's say, than this one over here. Does that make sense? AUDIENCE: Yes. CARTER ZENKE: Yes? OK, one thing I notice is that this student, they, I think, are asking the same question but in different ways. Yeah? This student is asking the question is the letter greater than or equal to 65, and is the letter less than or equal to 90? And that's confusing to me, at least, because how could a number or how could a character, like A, be greater than 65? Does that make sense? I mean, it does kind of because it can. Because we know that in C, letters are actually numbers. So I think that makes sense, but if I'm a human reading this, it takes me one more step to understand what they're doing. I have to convert the character to a number in my head before I know what's happening in their code, right? Whereas this one was a little easier. It's almost more like English. If it's alphabetical, I'll add one to my number. This one I actually have to look at the numbers and be, like, is it between this? Is it not between this? It's harder to read. Does that make sense? So I would say this one is maybe better designed than that one. Would you agree? AUDIENCE: Yeah. CARTER ZENKE: OK. YULIIA ZHUKOVETS: And maybe one last tiny, tiny thing that I notice is that this example declares variable L as a float, and the second example declares variable letters as an int. Which one is better, int or float? Integer, right? Why? Yeah, I'm hearing some good ideas. So I know that all of my letters will be whole numbers, right? There will be one or two or three. There can't really be 1.2 letters, right? And if I declare my variable as float, it takes up a lot more memory than an int, right? So that's like one tiny inefficiency that you can consider. So I would say that declaring variable letters, or L, would be better with the data type int rather than float. Therefore, I do think that example to the right-hand code would be better design. CARTER ZENKE: Yeah, so you've seen here three ways that these programs differ, and it seems like this one might be better designed because it made some things more efficient with those three ways, right? So to summarize, these were the three things we looked at in readability. There can be more things, too, but these are a few examples. So did the student use built-in functions, like isalpha, or did they reimplement that, for instance, with the numbers, like greater than 65 or less than some other number. Second one was does the student create helper functions, like count letters that help you understand their code, or is it all in the main function? It can sometimes often be better if a student creates helper functions, like the one we saw earlier, like count letters, right? Now the third thing is do they use the right data types? Like Yuliia said, in this case, an integer? Int makes much more sense than a float. So you could look out and see does the student use the right data types, OK? So these are three things to look for in readability, but there are probably more, too. As you see more student code, you'll see more things to look at, more things to compare across different students, OK? Yeah. YULIIA ZHUKOVETS: Carter, do you want to go back to the previous slide and maybe we can talk how to actually assign a score for a design, right? We told you that CS50 uses a scale from one to five. So maybe this would be a good example to maybe discuss what score you think each of these programs should be awarded. CARTER ZENKE: For sure. So I know that we said earlier that a five was perfect design, right? Five was perfect. Now do either of these seem perfect to you? No. I mean, there's probably improvements for each one. This one, I would say is pretty near perfect. I can't, myself, think of a better way to implement this one. This one though, we thought of some improvements, a few improvements, maybe one or two, three to four improvements, yeah? So I might say this one could be a three or a four on design, OK? This one though, I think it's pretty close to perfect, and so I might say if I'm being a little generous, I might say it's about five, right? Yeah, as an example, would you agree or would you disagree? YULIIA ZHUKOVETS: I think-- should I agree? Should I agree? Well, no, I think this is one. No I'm joking. [LAUGHS] All right. No, I think Carter got it pretty right. The right-hand code looks really, really good, right? We couldn't even find any improvements to make. But maybe I will point out one thing is that we don't see the rest of the code, right? So maybe their count letters function is perfect, but once we get to constants, it's like, boy. Oh no, they're using all kind of different things that I don't even know how to read. So I would say that these chunks of code might be awarded maybe a five and a three, respectively. But when you look at your student's code, you need to consider the whole picture, looking at all of the main function components and then looking at all the helper functions component and combining all of these things together. So for example, in this case, we found two or three improvements with just the counting number of letters part. But maybe there are some other things to consider in the other parts of the program, OK? CARTER ZENKE: And it is OK to disagree. So if Yuliia maybe disagreed with you, that would be OK. We can still be friends, right? We're still friends. And often, I think you'll find that somebody says this is a five, but somebody else might say this is a three or a four. And so you can talk about why you think differently, and maybe one of you will be persuaded to join the other side, OK? So let's look at one more example. This one is called Caesar. Do you remember Caesar? YULIIA ZHUKOVETS: Do you remember Caesar? Did we like Caesar? AUDIENCE: No. YULIIA ZHUKOVETS: No. OK, CARTER ZENKE: At least we're honest. That's great. All right. So we have a first example here of a student who implemented Caesar and submitted their code to you. YULIIA ZHUKOVETS: All right, so as with the previous example, we're seeing just a little snippet of it, right? So maybe I'll recap what Caesar was all about, right? So Caesar was a really a program where you had to implement some encryption algorithm, where you would shift a letter by some number, given the user's prompt. So maybe instead of main, what's going on is that we're taking user's input. We're doing some error checking, and then we are implementing maybe a FOR loop that goes through each character in the code and then rotates them by some number or shifts them by some number that the user inputted. And so this is exactly what we're seeing on this slide, right now, right? We have a function to rotate a character by n positions. Oh, this is a really good comment. It tells me exactly what I need to know, right? Even if I maybe don't know what the code actually means, I can guess, just from the comment that the user inputted. So let's look at the function. First thing that they do is that they specify the parameters or arguments the function is going to take in. So a character c, int n. I will speculate to say that c is just a character in the text and then int n will be the key by which we need to rotate or shift that letter. And then first thing that the student did, they checked if the character is uppercase using an isupper function. So if it is uppercase, they're applying a certain algorithm to shift that letter by a certain amount of positions. Next, if the first statement wasn't true, they're checking if the character is lowercase, right? Because the algorithm will be different for lowercase and uppercase letters. If it is a character and it is lowercase, they will go into a slightly different algorithm to shift the letter by n number of positions. And lastly, they will return the character unchanged if it's not a letter, right? So at every step of the way, they're returning some kind of value, depending on if the condition was true or not. OK, how do you feel about this code? Does it make sense to us? OK. So maybe Carter can walk us through the next example. CARTER ZENKE: OK, let's see an example from one more student. I'm hearing some murmurs. This is the same function rotate, OK? And the goal of rotate, you remember as Yuliia said, was to take in some character and rotate it some number of places. So if I have A, my key is one. What should I get back? B, right? Because one after A is B. What if I had A and a key of two? What should I get? AUDIENCE: C. CARTER ZENKE: C. What if I had an A and a key of three? What should I get? AUDIENCE: D. CARTER ZENKE: Good. If I had B and a key of one, what should I get? AUDIENCE: C. CARTER ZENKE: C. OK, I think we're getting it. That's good. So here, it seems like the function rotate takes in a character, like A or B or C, and some key, like one or two or three or four or five or six or whatever it is, right? And we're going to rotate the character and return back the updated character, like we just saw in Yuliia's function earlier, too. Now, how does this code work? Well, it seems like we start with a variable called norm, and it's type integer, OK? Do we know what norm does yet? Mm, don't really know yet, but we'll see. So then we ask a question. If C is greater than or equal to A and C is less than or equal to Z, norm equals 97. Does that make sense to us? I'm a little confused, too. YULIIA ZHUKOVETS: A little confused. CARTER ZENKE: I'm a little confused, too. That's OK. I think what we're doing here, again, often you will be confused when students send code to you. I am often confused when students send code to me, OK? That's OK. So we'll walk through it step by step. I think they're asking is the character lowercase? Is it lowercase? That's the question they're asking up there? And if it is, if the character is lowercase, well, we'll set norm equal to 97. Unknown reasons yet, but we'll see. So now, otherwise, if the character is what? Uppercase, right? So if it's uppercase-- we saw the case if it's lowercase. But now the next one else if C greater than equal to A, that will be if it's uppercase. OK, if it's not lowercase or uppercase, what do we do? It's probably not a character. It's probably a number, like a one or a two or a three. We're testing to see if it is an alphabetical character or not. If it's not, if it's like a number, like one, or a special character, like an exclamation point or a period, we'll leave it alone, OK? So we'll only take and work with these alphabetical characters. OK, now I see here, we're taking our character and subtracting norm from it. Mm. So if I have a character, like A, what does it mean for me to subtract a number from it? It's a little bit interesting, but remember that characters in C are numbers, too, right? So A is represented by 65. Capital A, 65. Lowercase A is what? Do you remember? AUDIENCE: 97. CARTER ZENKE: Oh, very good. 97. So it seems like what we're doing is taking our character and bringing it back down to zero. So if I had capital and I subtracted 65, I would get, in this case, zero. If I had B though, B is 66. What is capital B minus 65? Well, that's 1, OK? So it seems like we're bringing it down to a lower number. And the reason for this, I think, is that if I give you a character of Z and a key of one, what should I get? YULIIA ZHUKOVETS: A. CARTER ZENKE: A. So there's no letter after A. We have to go back around to the very first one, right? So I think this is helping us do that. Down below, we're going to add the key, modulo 26. This is also helping us not overstep the bounds, going past Z to some unknown letter that doesn't exist, right? We're going to go back to A. And then we're going to ask the question here. If C is greater than or equal to 26, let's subtract 26. I'm not sure why we're doing that friends. That's OK. And then we'll go down here, and we'll add some character to norm. And we'll return the character, and I think maybe we'd get back the right character. All right, so that was our overview of this example here. Let's compare them now. And we'll actually ask you to do this. YULIIA ZHUKOVETS: All right, so in the previous example, readibility, you saw me and Carter kind of go back and forth, discussing some differences between example 1 and example 2 and maybe discussing what improvements one or the other could make. So what we want you to do now is turn left or right to your peer next to you. So maybe if you're on the edge, on this edge, turn left. If you're on that edge, also turn left. OK? Anyway, let's pair up and talk to each other in pairs. Which one of these codes is better, maybe what improvements one or the other can make? And then we'll come back all together and discuss again, all right? So let's turn next to each other, make pairs, and discuss the code, all right? [BACKGROUND CHATTER] CARTER ZENKE: Oh, very good, friends. Thank you. Thank you. Thank you. OK, so I thought we'd do something a little unconventional here, which is we've now seen our two student submissions, yeah? We're going to cheer for the one we like the most, the one we think is the best, OK? So if you think this one was the best, please cheer for me. [CHEERING] OK, OK. That was pretty good. If you think this one was better, please cheer for me. [LAUGHTER] OK, one more time. If you think this one was the best, please cheer. [CHEERING] If you think this one's the best, please cheer. [CHEERING] Oh, OK, OK. OK, so it seems like we think this one might have been better designed than the other one, OK? [CHEERING] All right, so let's now ask why? Why is this one better designed? Yeah I'm hearing a few ideas. So let's go ahead and raise our hands and see. Let's go maybe back there Yuliia? [BACKGROUND CHATTER] AUDIENCE: OK, I would like to tell why I chose the left side one better is because the student is make themselves clear by using common. So we can understand, line for line, what did they do. And also, they just use the character A, not the number. Because I remember from one of the lecture that maybe we somehow forget what is the number for A? What is the integer for lowercase A? Or is it a 66 or 65? So it's better to use just a lowercase A and uppercase A so will decrease the possibility to have a bug. And also, the last code of the right side is a little bit complicated that maybe we will forget to decrease the norm or add something to the key. So it will make the possibility to be a bug is bigger. So that's why I chose the left one. CARTER ZENKE: Yeah, thank you so much for sharing. So to summarize, I think you have a lot of great ideas. Some improvements over here is that the student used comments. Oh, thank goodness, right? Comments are always helpful for us, as teachers. They also seem to be using characters instead of numbers, which is helpful, as you said, because sometimes we forget what numbers equal what characters, right? And another point that I like that you said, too, is that this one is pretty complicated. It's pretty long, which it's OK, but it also can lead to more bugs, right? Now when we talk about bugs, we're really talking about correctness issues. Does it not work in some way? So if you have code that is not well designed, it can also lead to correctness issues, like bugs. OK? OK, let's see other ideas, too. Any more reasons to choose this one over that one? Yeah, let's go there. AUDIENCE: In addition from the previous participant is that in the right code, you have to go through all the way down to return the C. When on the left code, if the first condition is true, you have to return without going through all the codes in the function. CARTER ZENKE: Yeah, I really like that idea, too. So it seems like on this left-hand side, we return from the function, we finish it much faster, right? So if this first condition is true, we return immediately. If it's not true, we check one more question and we return immediately, right? There's not a lot of code to go through, depending on the character we have. In this case, like you said, whatever character we have, we have to go all the way down through this code and finally return C. So a good observation there about why that one might be better designed. Good. Any more ideas? OK, one more. AUDIENCE: Thank you. We are laughing because yesterday, we are in back then. We are always hands up, but we are-- then today we are here. And no, no, this way. Thank you for the opportunity. OK, for the first one, I think it's better because it used isupper, islower. So it's much better than the right one, yeah. Then when we look at the code, the rotate is always repeat. repeat, again and again, so it takes time. And the code in the left side is better. CARTER ZENKE: Yeah, I agree. AUDIENCE: So we decide this is five and this is, because we are good teacher, four. CARTER ZENKE: Thank you for sharing. [LAUGHTER] OK, so I like the ideas here. I like, in particular, that you mentioned this one uses helper functions, built-in functions, like isupper, islower. We're not remaking, reinventing the wheel, so to speak. We're, instead, using what C gives us in terms of the functions, as opposed to writing our own code to do exactly what R has already done by isupper and islower, OK? Good examples here. YULIIA ZHUKOVETS: Yeah, and so I think we already got some suggestion on the design score, and maybe we can actually take a vote on what people think this code should be awarded. So maybe if you think that the left code should be awarded five. Cheer now. [CHEERING] OK. And who thinks it should be four? Who thinks it should be-- OK, I hear some faint cheer. Who thinks it should be three? [APPLAUSE] OK, so maybe we agree on somewhere between five and four, because this looks pretty good to me. But remember that we don't know what's going on inside of main. Main can be bad, right? So we need to look at the whole program to decide what score to award. And then, let's take a quick poll for the right example. So who thinks it should be awarded a five? OK. Silent. No one. OK, not perfect code. What about a four? [LAUGHTER] OK, OK, I hear some laughter. What about a three? [CHEERING] OK, and two? [FAINT APPLAUSE] And one? OK, I heard one clap. So I think we can agree here, friends. So maybe if you're feeling very generous, like these teachers over here, you can give a four. But really, it should be around a three, maybe even a two if you want to be a bad cop, right? So all right, so very good thoughts. Very good ideas. So now, we will turn things over to Carter so that he can talk about some of the things that I and him thought about before this session. CARTER ZENKE: So very similar to you all, we thought of a few things we would look for in this program here. To other people's points, we looked for if the student used built-in functions, like isupper and islower. We also asked did they use an efficient method for converting the character? To the point we had earlier from teachers here, here, we argue, is pretty efficient. The other one, not so much. OK? So just a high level overview of what you can look for in a program like Caesar. But now-- YULIIA ZHUKOVETS: Now we're actually going to turn things over to you, and you will do exact same process that Carter and I did on the board twice now but with a code example that you will have on the tables in your breakout rooms. And I will turn it to Carter to explain the specific instructions that you should do. CARTER ZENKE: OK, so in just a moment, we're going to leave the room, but not yet, OK? Not yet. When you leave the room, you'll go to the same table you did microteaching at yesterday. You had homework. Do you remember your number? OK, thank you. When you get to your table, you should find three of these papers, OK? Now this paper is very important. It has two sides. This is one side. This is the second side. First side. Second side. Paper is two-sided, OK? Yeah, now, in the top of it, you should see Sepia1.C. Sepia1.C on one side. On the other side, you'll see Sepia2.C. Do you remember Sepia? Sepia in the filter problem? Did you like the filter problem? OK, some mixed reviews. That's OK. In the filter problem, you wrote a function to take an image and apply what we call a sepia filter. A sepia filter makes things look a little yellowish, if you remember. OK. So here, we actually have two real student submissions for the Sepia function, not the entire filter program, but the Sepia function. On the first side, Sepia1, you'll see one student, OK? On the second side-- remember, it's two-sided. This is the second student, OK? First student. Second student. What's this one? AUDIENCE: First student. CARTER ZENKE: What's this one? AUDIENCE: Second student. CARTER ZENKE: Two different students. OK, So here are your instructions. What you'll do is this. You will, on your own, individually, you will look at the paper, look at both submissions. Oh, in pairs, sorry, you'll be in pairs. Find somebody next to you and pair up and review it together, OK? You will then choose a design score, like we just did, one through five, for each student submission. OK? Now, remember, just like we did here, it's very important to walk through step by step and make sure you understand the code. If you don't understand it, you can ask somebody next to you and see if they understand it. And if they don't understand it, well, that's OK. You can talk through it, OK? You can try to understand. Talk in your group and figure out if you can understand it all together, OK? So for each of these submissions, you will do what? Look at it individually, and then give a-- AUDIENCE: Grade. CARTER ZENKE: Grade, a design score, one through five. All right, once you have your design score, one through five, you should discuss. You should discuss because maybe I think it is a five but my friend thinks it is a four. We have to agree, OK? Ideally. It's OK if you don't agree, but ideally, you want to get to a place that you agree, OK? And once you do, go ahead and do one step more. Go ahead and write on your paper with a pen some comments for the student. Mm, because it's not helpful if we just give a score. If I just write a five, what does that mean? If I just write a four, what does that mean, OK? So write some comments on the paper, and tell the student what they could improve or what they did a good job on, OK? So think of this as your own student who has submitted work to you, as a teacher. All right, one more time in Bahasa, Indonesia. This is what we'll do. Again, you'll find three of these papers on your tables. How many sides are there? AUDIENCE: Two. CARTER ZENKE: Thank you. There are two sides. For each side, you will give a design score, one through five, after reading it and understanding it. After you give a design score, you will try to agree with your entire group for each problem. And then you will write some comments for the student on each side, OK? What questions do we have? Make sense? Does it make sense? OK, that's good. So we're going to follow our algorithm here once, again. Now here is how our algorithm will go. If you are in groups 1 through 10, please raise your hand. Groups 1 through 10, so good to see you again. Could you get ready for me? Could you stand up? YULIIA ZHUKOVETS: Let's just take five minutes to recap what we just did. So we asked you to discuss the code, assign it a design score, and maybe write some comments with improvements. So I thought we can take this time to do a quick poll on what design score each group awarded. So first let's take a look at Sepia1, Sepia1 with some IF statements, so the longer code, the one was on the first side of the page. And so if your group awarded a design score of one, cheer now. One. If your design score was two, if you're-- OK, some pain sounds. If your design score was three. [APPLAUSE] OK. And if your design was four. [APPLAUSE] Woo. All right, and what about five? OK. [LAUGHS] All right. OK. So I think we agree on somewhere between 3 and 4. So now let's take a look at Sepia2. So to me, this code looks a little bit more simpler and succinct, but I want to hear what you, friends, think. So if your team awarded this program, Sepia2, one, cheer now. One. What about two? [LAUGHS] What about three? OK. What about four? [APPLAUSE] OK. And what about five? [CHEERING] OK, it seems like folks thought that Sepia2 was a little bit better design, and I think I would agree with that. What I really like about this program is that it looks, first of all, shorter to a human eye. Second of all, it makes use of some built-in C functions, right? So it uses round and fmin. And I don't really see any repetitions, right? The code is pretty simple and algorithmical. But if we go back to Sepia1, one thing that really stood out to me was these IF statements, right? So to me, it looks like the student might have just copy pasted it three times and really, if we find ourselves copy pasting things, we should probably use a helper function, OK? Do we agree on that? AUDIENCE: Yeah. YULIIA ZHUKOVETS: Yeah. OK. So one last time, if you think that Sepia1 is better designed, shout now. [APPLAUSE] OK, and if you think Sepia2 is better design-- [CHEERING] All right, great job, friends. And as you've seen today, it's really a lot about comparison and contrasting. So as you find yourself grading more and more problem sets and programs like this, you'll find yourself being able to award more consistent design scores and leaving some good comments for your students. But for now, this was a session on grading and feedback.