DOUGLAS KIANG: Thank you to Carter, and Bernie, and David for inviting me here to share what I do in my classroom. I teach at Menlo School, which is in the Bay Area in Silicon Valley, but I recently moved from Hawaii, where I taught at Punahou School. So a lot of what I'm sharing I did back in Hawaii, and now I'm trying to do this at a different school. But they're both private schools, and you know I teach at the high school level. So I teach probably two to three sections of CS50 depending on enrollment, between 14 and 20 kids per section. And a lot of the tools that I'm going to talk about that the CS50 folks have developed have just really made it possible for me to teach this course. It would be hard to manage without things like check50, and the online submission platform, those kinds of things. 

I think Carter sent out links to other presentations that they've done to do sort of a walk through of those tools. I'm going to just show you what I do, but not necessarily how I do it. For a walk through of how to use a lot of those tools, you can look at some of these other videos and things. But with that said, let me share a little bit about what I'm doing. And what I'm planning on talking about is I'm planning on talking about these six things, hopefully I can get through all of these. And I'll be hanging out during the reflection if people have other questions or follow-ups, and certainly post things into chat as well as you have-- as you have questions. 

So let's talk a little bit about how I structure a lesson. So when I introduce a new computer science concept, I think about in three parts. I introduce the concepts through an unplugged activity, I introduce the skills through a bird-- what I call a birdhouse activity, or a guided activity, and then I'll usually try to give the kids some kind of an independent project to show that they can apply those skills. And I do this in all my CS classes, whether it's CS50 or anything else. I just find that this framework works really well. Let me tell you a little bit about what each of these steps is. So an unplugged activity, as you probably know, is an activity that generally takes place traditionally, not on a computer, but of course now we're all on computers all the time. 

So-- but I try to-- I find that when kids sit in front of an IDE, a lot of the time they get very-- they get tunnel vision, on the code and on the syntax. But it's really important for kids to get the concepts first. And so as an example, if we're talking about sorting. We have an array of numbers, and we have an unsorted array at the top, and we want to move from there down to the bottom where it's sorted. I usually will start the conversation by just saying, well, how might we go about doing this? Give kids a bunch of stack of papers, or other objects have them-- how would you sort them? And what's interesting is most kids-- and most teachers-- if you had a stack of papers, the way you normally would sort it is probably insertion sort. You take each piece paper out of the pile, and you put it where it needs to be in the sorted pile. 

But the first thing, of course, we teach is the pseudocode around bubble sort, because it's simple and it's kind of easy to get your head around. But it's not the way that a human being would naturally sort anything. And so what I'll often do is I'll use gallery view in Zoom. What I used to do in person, when we were face to face, is I'd have the-- I'd pick like eight kids, and I'd give them a sheet of paper with a number on it, and they'd stand at the front of the room. But since we've been on Zoom, this is one of the few things I found that Zoom actually, I think, does better than face to face. Because you can grab-- if you go to the upper right-hand corner, and you see that speaker view, gallery view. If you put yourself in gallery view so you can see a grid of everybody, you can try just dragging somebody's head-- and try dragging them around, you should be able to. 

Or at least, in my school, on my Zoom, you can drag people around. Maybe it's because I'm not the host or something. But you can-- I find that you can actually talk about, OK, let's take the first two people. And for me it's [? Ahmad ?] and myself, and he comes before me because A comes before D. So we're going to stay where we are. Then it's Douglas and Carter, Carter is going to come before me because C comes before D, so I would just reach over and swap the two of them. And I can have the students practice each of the sorts this way. And once they've done that, then we can go on to selection sort, go on to insertion sort. So that's an example of an unplugged activity. 

And when I do it in person, it's always tough because the kids who are standing in front of the classroom holding the signs, they don't see the sort. So I've got to run it a bunch of different times, and pull different kids up and convince them to go stand in front of the class, and it takes longer. But I found this activity is actually quicker on Zoom in gallery view. And then, there used to be a feature-- I don't know if they've hidden it, Zoom keeps changing things-- but there was a way where I could force my order to everybody else, I could push it out to everybody else. So everybody else sees the order. So I would have kids do their sort, and then push it out so that we could see it. So that's a unplugged activity. 

Then we go to the birdhouse activity. And a birdhouse activity-- you know when I was in school, we had wood shop, and everybody made a birdhouse. That was one of your first projects, right? And the thing about a birdhouse, everybody makes the same thing. They give you the plans, everybody does the same thing. Why are they always the same? Because it's easy to assess, right? You know what it's supposed to look like. So if someone turns in a birdhouse, and the roofs a little crooked, you know OK, you've got to work in your cutting skills, your measuring skills. It's easier to assess skills when everybody's doing the same thing. 

The problem with most schools is that they stop after the birdhouse activity. They say, OK we taught the skills, we assessed the skills, we're good. Let's move on. The problem is, with a birdhouse activity, you don't ever really get a sense for, can they apply that knowledge in a new context? If they can regurgitate it to you, OK they've memorized it. But they haven't really learned it until they can demonstrate that they know how to call on those skills in a different situation. So later on in the course, we teach Python and dictionaries. So I use CS50's sandbox tool to create what I call birdhouse activities, or guided activities, where everybody does the same thing. 

So here are some examples of prompts. I give them a link to some content, hey go look at these dictionary-- go read up about dictionaries if you want to learn about dictionaries in Python. Then create a car, I give them a line-- line 3 is an example of how to construct a car dictionary-- and then I just give them a bunch of things to work on. OK, I want you to print out. Go through there and print out the description of the car, And then create three more cars. And then create a list called garage and it contains all of those cars. And then, now all of a sudden, hey, you've got a brand new car! You get to choose all of that information, now add it to the garage. And there's a bonus if you can create that car and add it in a single line of code. Not all kids can do that, some really relish that challenge. And then iterate through the list, and print out, OK, I have a whatever for each car. 

Now these are the steps that kids need to do for DNA and for World Cup, which are two Python activities in the CS50 curriculum. And when kids get stuck, I always tell them go back to when we did the cars. You remember the cars? Do you remember what the garage was? The garage was a list of dictionaries, how did you assess the value for a given key in one of the dictionaries in the list? And we talk them through that. But the way that I think of the problem set is it's the application of the skills that we've taught, through the unplugged activity and the birdhouse activity. 

For the sorting thing that we just talked about, what I might do after I did the sorting in gallery view on Zoom is I might take them to a sandbox and say, OK. I've already created these prototypes, and I've even created a way to print out the contents of this array. But what you need to do is write the bubble sort algorithm that's going to sort these things. And we'll do this together as a class. And then later when we go and we do a problem set that involves sorting, they already kind of have the write code, but they have to tweak it and change it a little bit. But the sandbox is really helpful, because you can go back to sandbox and you can see all your recently used sandboxes. So I'm always encouraging kids to go back to the CS50 sandbox, try out their code there, try out examples. 

The one thing I do insist that they do, is they go to the down arrow and name all their sandboxes. If you don't name the sandbox it ends up with a random name, and it's really hard for kids to look through that list and figure out what it is. So I always tell them-- you train them from the beginning-- the first thing you do, name the sandbox so you know what it is I'm working on. So birdhouse activities like I was talking about. These are guided activities, you're walking kids through, you're giving them the code they are learning these new skills. So they're really high on new skills acquisition, but the problem is they're really low on creativity. Because you can't really do that much with a guided activity, right? 

Like if you're making a birdhouse, how do you differentiate your birdhouse? You might paint it a different color, but that's just a superficial difference. You don't really know if they have the cutting and the measuring skills really solid until they apply them in a new way, or demonstrate their understanding in a different way. And so that's why I give them an independent project. And an independent project is very low on new skills acquisition, because I'm not asking them to gain new skills. What I'm asking them to do is demonstrate them in a creative and meaningful way that's personal to them. And that's why independent projects and birdhouse activities go hand in hand. 

Some schools just skip the birdhouse activity entirely. They say, hey, we're going to learn to make an app. Go teach yourself, the internet's a wonderful place. And the problem with that is, how do you assess the skills? You don't know what skills they have. They might be able to do it, but they're learning from YouTube, or wherever. It's all over the place. So you can't really assess where the gaps are unless you have birdhouse activities-- guided activities-- first, and then you have them demonstrate them. So that's the application of it. So one of the things that I did is I said, OK, here's a challenge. Now that I've introduced you to sorting through Zoom, we've done some sandbox activities in Zoom, now what I want you to do is teach somebody how to do one of those sorts. 

And this is going back to what David was talking about, making the learning visual. How do you make it visual? Well my challenge to the kids is, you make it visual. You show me that you understand this by demonstrating what one of the sorts looks like. This person did stop motion, so chose a little Minecraft cow to kind of show-- so this is actually selection sort. And what I really loved about this was all the creative ways that kids interpreted this assignment. They did it in all different kinds of ways. And they did it at the level of comfort that-- some kids didn't want to talk, so she didn't use sound at all which is fine. This person decided to illustrate insertion search and kind of edited the whole video together, put captions. So the kids just really took off with this assignment. 

[MUSIC PLAYING] 

So when I ask myself, do they understand this? It's this kind of application that kind of tells me, oh, OK, they get it. They get the steps, and then they can apply it in a different context. And what I did is, I actually took some of these videos and I put them on one of the quizzes, because we have regular quizzes in my class. And I just took the sound out and I just showed the video and I said, can you tell me what kind of sort this is? And the kids should be able to recognize by looking at the movements, when they really understand it, what kind of sort they're looking at. 

All right, so that's lesson structure. I'm going to talk about planning the year in general, and in that folder what I have done-- let me go ahead and stop the share for a second here. I've given you a link, and I'm going to put this in chat here, this is a link to a shared drive folder, OK? And in that drive folder, I've got-- I've got kind of a-- my own unedited planning document that I put in here. So this-- I apologize for all these, they're just notes that I made to myself-- a lot of this stuff, some of the links don't work because they're internal links to my school, but I just wanted to give you a sense overall of what does the first semester look like. 

We start with Scratch, we have them do a little mini independent project, then Scratch, and then I just keep a Google Docs. And I don't share this with the kids, the way the kids know what to do is they go on Canvas. But this just reminds me what to do. And when I find links, or resources that I want to show, I just paste them in here. And I paste some notes to myself, and sometimes I do the class and it doesn't work really well. But I know I'm going to do it again. So I just make notes to myself of what I'm doing. So you know, kids I need to check in with in class because they're missing stuff, I kind of throw that in there too. A lot of the sandboxes where I'm doing the guided activities, the birdhouse activities, I put links in there they're all bit.ly links. So if you want to see some of my sandboxes, they're all here. 

But again, I apologize that I didn't organize this better for the workshop. This is really how I use it, and sort of in all of its messiness. But this will give you a pretty good sense, and that's the first semester. And then I also have the second semester here. So the only thing I will ask, just for privacy's purposes, is this is for us. So don't share this out on social media or anything like that. This is just for you and me. So-- if that's OK? All right. So let me go back to-- let me go back to my slides here. So I want to talk a little bit about some of the details. That will give you a sense of what the year looks like for me. 

But what does the whole four year process look like for students? For all the students, the intro level course CS50. So any student who wants to take any CS class at Menlo School has to take CS50 first. And the way that we explain that to kids is we say, look, it's kind of like-- because a lot of kids, they go to summer camp and they're like, well, I already know Python, or I already started Java, so why can't I skip CS50 go on to CS A? And what we tell them is just because you can drive a car, doesn't mean you know how it works, right? Like one of these higher level languages, like Python and Java, it's kind of like driving an automatic car. But CS50 is like learning stick shift, and going in under the hood, and seeing how the engine works. 

Because we think that there is real value in understanding, at the level of bits and bytes, how computer science works. It's not enough just to know how to drive the car, you need to know how the car works. And when I explain it that way, kids-- many of them still don't get it, the parents do though. But that's a conversation with them about why we have this prerequisite. So at Menlo, all the middle schoolers have to take computer science, and it's all pass/fail. And so what we lobbied for this next year is that CS50, our intro level computer science course, is also credit/no credit. Which is a sea change because there are no other credit/no credit courses in the high school at Menlo. So we're going to be the first credit/no credit course. 

But I was inspired to do that after seeing what CS50 did at Harvard, making their course basically opting students into credit/no credit. We're going to do the same thing. We will give kids the option for grades, but what I've found is that the fixation on grades at schools like Menlo and Punahou is not great for learning something like computer science, because kids tend to look around at each other too much and start making too many judgments about their fitness or their readiness for it. I mean this is a new language, and I'm used to seeing kids struggle in the beginning before the curtain goes up and they're like, oh, this makes sense. That might not happen until the second semester. But I'll have kids who just won't even try computer science because they're afraid of getting a B-plus. They just won't even try it. 

So we're-- so next year, first year, it's going to be all credit/no credit. We went to-- we got-- we broke enrollment records this year by doing that. And we gave surveys about how important is the credit/no credit, turns out it's pretty important to kids. It convinced a lot of kids to take the course who would not otherwise have taken the course, and we have the data to show that. So that's why everybody comes in at CS50, and after specifically they have two choices. They can go to AP CS A, which is in Java, or they can take an app design course. We follow Apple's develop in Swift curriculum, which uses Xcode and Swift, and we'll have kids actually creating apps. After app design kids would normally go to CS A, but with permission they might go on to Advanced Topics. Advanced Topics really covers much more about data structures, and more than mathematical foundations of computer science. So it's fairly rigorous, so generally the progression would be you take CS A, then you go to Advanced Topics, and then we also have an Independent Study. So that's kind of how we've got it organized. 

So in terms of planning the year, I'm going to-- I'm going to go through the chat afterward. I have a lot-- I've still got a bunch of stuff to get through. So if you'll bear with me, I'm going to circle back on those questions. I found that TAs are very important. The way that I convinced the school to go credit/no credit, part of what I said was, CS50 is not an easy course. It is a challenging, difficult course, especially for freshmen. And it's intentionally so. But my intention for an intro course is not to dumb it down and make it super easy just because we want to bring kids in. I think that kids can handle a rigorous curriculum if they have enough support, and CS50 has a lot of really great scaffolding. 

I think the shorts are really important, because some kids won't watch the lectures-- sorry, David-- despite what I do. But they'll watch the shorts, and they love the shorts. And other kids, they don't get the shorts but they love the lectures. So the more different ways you can provide information to kids about what they need to do, the better. And everything is built around the problem sets. I have TAs now, so I bring in TAs, I work very closely with students who are TAs who can work with other students. I showed you the Google Doc. We have them work on a new problem set about every week or so, might be 10 days if it's a difficult problem set. Larger problem sets I'll break up into parts, and I'm giving them regular feedback on the problem sets. 

For the most part we have them watch the lectures outside of class, that's their homework. In class they're working on the problem sets. So they come in, they work on the problem sets, but they're expected to have read the problem set and have watched the video when they come in. One of the things that has really helped a lot is I have been using Edpuzzle. And Edpuzzle is a great way to-- you'll know if kids watch the video or not. So I've actually taken a bunch of David's lectures and I've broken them up into about 20 minute chunks, because that's about as much as my high schoolers can watch at a given time. They'll watch a 20 minute portion of the lecture, they'll answer three or four multiple choice questions, and maybe one reflection question. 

And then I can see immediately who's viewed the video, and how long they've spent watching the video. Edpuzzle allows you to set up whether they can skip backwards or forwards, or what-- it gives you lots of tools. So I'm going to have my TAs convert all the lectures that I assign into Edpuzzle videos and write multiple choice questions for them, because I found that that's been very effective. And I've actually heard from my own students who have said, I tend to get distracted a lot, so thank you for putting these into Edpuzzle because it helps. So lectures are homework, in class is when I would normally walk around and meet with kids. Yeah. So Edpuzzle is-- someone can throw up a link to Edpuzzle-- Edpuzzle.org I think, or Edpuzzle.com. But it's free. 

Yeah, you know what? Hold on, I'll just show you. So yeah, I was stoked when I discovered them. Actually, my wife showed this to me. She uses it, she teaches CS in middle school. So you got to EdPuzzle-- yeah, EdPuzzle.com I guess is what it is. So I've taken-- so here is, let's see. Yeah, so here's the intro to Python. And I basically took one of his lectures, and you can just-- they can play the lecture from here. And every so often, it's going to stop. And it asks them this question. And the question is basically, did you watch the video, and can you answer this question? 

And often, I give myself permission-- like I love to get all the way through the CS50 curriculum, but it really depends on the class. Some years, we just don't get there. Some years we just go as far as Python and we don't get into HTML, CSS, and that's OK. This year I actually stopped a little earlier, because we covered less this year than I thought we were going to cover. But, you know, it's OK. We do puzzle day, I can't stress enough how important puzzle day is as a way for kids to come together early in the year. I know Angelo will probably talk more about the real importance of building culture and community, but I know earlier somebody in chat was saying, I'm finding it really hard to learn this stuff. I agree, this is not easy. But it is easier when you're with other people. So to whatever extent you can find other people to learn this stuff with, I think the easier it becomes, because you can support each other and people get stuck at different places. 

We do a hackathon, we collaborate with our local ACM group at one of the universities and they hosted our kids over. And these are college students helping the students out. We gave them time to work on their independent projects, and they kind of felt like they were grown-ups because they were at the University. This is submit50. And submit50-- this is submit area, and this is kind of where I manage all of my assignments. So every time kids hit the submit50 link it ends up here, and I can check it. And so I can see how many kids submitted, and I can go in there and I can leave comments for kids. And so what I try to do is as soon as the kids turn this stuff in, I go through there and I give them comments. I give them in-line comments, and I also give them a summary comment at the end that basically says, here's where you're doing a really nice job. Here's where it maybe was falling down a little bit on correctness, or here are some ideas for design. 

It's a lot of work, especially if you've got a lot of kids, but it's really, really important. And what we know about learning is the sooner you give the feedback, the more effective it is. I found if I wait a week, or two weeks, to give feedback on a problem set, they're already working on the next problem set. So it's kind of pointless. So I try to just get on that as soon as I can. That's my biggest challenge, frankly, is giving timely, effective feedback. But I also know that it is one of the things that has the biggest impact for kids. And I actually keep notes on everything that I write, so every time I write a comment, I actually paste it into these text files. I have a whole folder full of text files with all the comments I've ever written on one of these problem sets. And so I start to define for myself what they are. 

And so for instance with Caesar. I have in all caps if they use an array, they used a single line of code to solve it, they have magic numbers in there, and they called isalpha, and they're just printing the characters out, then this is the general comment that I use. And I just copy and paste that in, and then I might tweak it a little bit for some of the individual things that I've seen. So I just have pages and pages of-- so I rarely have to write a comment from Scratch anymore, because I mostly have already got-- I've already seen something that was similar to what I saw students do, then I can paste in the feedback that way. Yeah, so that's a suggestion that I have for you, is any time you write these comments, just save them in a text file so you can reuse them. 

Final exam after the first semester basically is in two parts. So we have a multiple choice short answer, and then we also have two free response questions. And what I do ahead of time is I give the free response questions out ahead of time, because my goal is to have kids really think through it and work on it in their IDE, and I ask that they not-- that they not work with each other on that. And then I also have to do a multiple choice exam. So here's an example of just one of the questions that I gave them. And the whole exam is actually in that shared folder, but if I put in-- create a program where if I said calendar, and then I gave the day, the first day of the week, and the number of days in the calendar, it should print out each thing. And they have to use modulus to be able to do this and it's challenging, but a lot of kids said it was fun. They had a fun time with it. 

And then the quiz is actually on Canvas. And I worked pretty hard this year to try to come up with what I consider a cheat proof quiz, which is really hard. But the way that I did it is I did exam questions in banks in Canvas, and I separated them into easy, medium, and hard. And so the quizzes that kids got were randomly assigned because they picked five questions randomly from each bank, and then I had some that I gave to everybody. And then I randomized the order of the questions, and then I made it so kids could only see one question at a time. And the questions themselves-- what I realized you can do in Canvas is-- I didn't want kids copying and pasting into their IDE or into sandbox to test these things out. 

So I actually realized that you can go into the HTML. And I embedded little characters, like an equal sign here, but I made it really small, like one point, and I made it appear invisible. And so there are invisible characters here that if you copy this whole thing, you copy it and you paste it in, what you get is slightly different from what the actual code is. Which is- my wife at some point pointed out this is kind of obsessive. And the thing is, what I found from working with students is that I think most, if not all, students feel a temptation to cheat. And most of the students don't. And they go through the pain and the hardship of figuring it out, because they want to do it honestly. About 10% of my kids will cheat, they'll succumb to that temptation. All of this work that I do is for the other 90%, because I want to honor those kids who faced the temptation to cheat, and still did it honestly anyway. Because fortunately most of the kids do that. And then the kids who do cheat, I've got to ding them on it just to keep everybody else honest. 

And so I'm going to talk a little bit more about that in a bit. But this is why I do a lot of these things. And so this-- so I felt that this quiz was fairly fair, but it was challenging for them. And I got a lot of these from the AP CS principles question bank, and I kind of rewrote them in C. So check50 is a great tool, kids use check50 regularly. Be careful if kids become over reliant on check50, what I found is that kids sometimes are not even running the code at all. They're just-- they're just running it against check50 and trying to get it to past check50, and they're not thinking about, well, what are some unit tests that I can use that will test this on either extreme of what could come in? And I want kids to be doing that kind of thinking, so it takes a little bit of work to kind of encourage kids to think about that. 

Style50 is an awesome tool. When my kids go on to AP CS A and they're writing in Java-- what's really interesting is the kids that come through CS50, their code is beautiful, because they've learned-- most of these kids it's the first time that they've used a text based programming language, and after they use style50 regularly they just start to write with proper indentation because they know what it's supposed to look like. And when I used to teach CS A, I used to teach beginners CS A, the indentation was just horrendous. Trying to train kids, and trying to break bad habits was really impossible. But now these kids are coming up from CS50, and that's why it's a prerequisite course for everything else. So we know that if they use style50, their style is going to look great. 

So this is what I see in CS50 IDE. One of the first things that I ask it to do is ask them to share their workspace with me. And when they share their workspace with me, what that allows me to do is I can jump into their workspace and I can see what they're doing. And I can even work with them, and you can collaborate in a given workspace kind of like Google Docs. So you can see what people are typing, and I can type in code as well. And so it was really effective this year for me to do a Zoom call with kids, but then either I share my screen, or I just pull up on my browser their code, and I can see them working in their code and we can have a conversation about the code while they're on it. 

It worked great with some of the shier kids who hate being on Zoom, they hate being on camera, because we're focusing on the collaborative code and we're basically looking at the same piece here. So that worked out really, really nicely just as a way to support kids. And a lot of the time, kids could just jump in and say, here's where I'm stuck, and I can talk them through it. And literally it would take like 5 or 10 minutes, and they're like OK, thanks Mr. Kiang, I'll see you in class, and then they hang up. Whereas normally, in a regular school, they would have had to set up a one hour meeting or something to come in and meet with me. It just, I find that I can make better use of my time because I'm able to share stuff through the CS50 IDE. 

The other thing that it allows you to do is, if you go to File and you choose View Revision History, you can actually scroll backwards through, and you can see what kids have typed. And you can see the process of their thinking, which I think is fascinating. You can, of course, also see if some kid had nothing and then all of a sudden, boom. They just have this entire passage of code, you know that they pasted it in from somewhere else. There's no way, no way they could have written it in two seconds, right? So you can see that stuff too. But that's not the primary reason why I use the revision history. I use it to support kids learning. But it's pretty effective. 

So that brings us to talking about academic honesty. Like I said, some kids are going to cheat. These are their solutions to most of these things available just all over the internet. And so I have the talk with them from day one about you are going to face that temptation, it is normal. I face the temptation too. If I pick up a puzzle, part of me wants to just go to the solution because I want to solve the darn-- I want it to be solved. But the joy is in the solving of it. And so I always tell them #NoSpoilers, you're giving yourself a huge spoiler if you look online. One of the things about a lot of the problem sets is that the problem sets-- people will post their solutions, but people will also, more and more, they'll use YouTube videos. 

So that they'll talk through how to solve it while you watch it up on the video. And they're pretty good, I mean, some of them do a better job than I do at teaching this stuff. They're really good. I've got to tell the kids, look, if you're in my class, and you're turning this in for credit you can't use that video. If you want to go do a problem set that we don't cover, that's a great way to learn. Good for you. You're going to YouTube, you're being a self advocate for your own learning, et cetera, et cetera, but you can't do that for the class. And I've got to draw that line ahead of time, because it is a pretty normal thing that kids will just jump on YouTube to learn how to do something. But kids-- usually it comes down to poor planning. Like they just-- poor time planning, they just don't leave enough time for them to do the problem set. So to get it turned in on time, they'll cheat on it. 

So what I do when kids turn in stuff is I take all of their submissions, and I run them through-- I run them through basically a checker called Moss. And what Moss does is-- it's run out of Stanford, and what it does is it goes through and it will compare, and it'll look for similarities in kids' code. And so this will show the similarities between what they worked on. And you can exclude starter codes so the starter code won't show up, but some of this stuff you can tell by what they chose. You can tell some of the variable names, whether they were working together. I tend to be more lenient-- oh, I think it was probably this. This is almost the same. I tend to be a little more leaning on kids who are working with each other, because frankly I do want them to collaborate. 

What I really come down hard on is kids who actually copy stuff from the internet. And so what I use to differentiate that is I actually have a-- I have a source. I have a GitHub, a private repo on GitHub, where I basically have collected, somewhat obsessively, over the years every single published solution to any of the CS50 problems sets that I use, and they're all in here. And the way that I organize it is I use a tilde to preface anything that comes from the net, and inside here there's a weblog that actually has the original URL at the time that I found it, as well as the actual code itself. 

So when kids cheat, and they find something on the internet, chances are it's in this repo and it will get found. So I have a database of anything that appeared on the internet, as well as anything that any kids have turned in previously for the assignments. And in that case-- is in those cases, sometimes they get kids who are older siblings, they took the course, they'll just-- kids will just reuse their solution. So that will get flagged as well. 

SPEAKER 1: Can I just read you a couple of comments? You're getting some really great thank yous. 

DOUGLAS KIANG: Oh yeah, sure. 

SPEAKER 1: The immaculate planning and dedication Douglas has put into mesmerizes me. Kudos to the hard work, and thanks for all the great ideas that you're willing to share with us. 

DOUGLAS KIANG: Wow that's super nice, thank you. That's very kind. That's very kind, thank you. 

SPEAKER 1: And some will reach out with their new emails, they're moving to new schools and want to get access to your resources. 

DOUGLAS KIANG: Yeah, no problem. Happy to share. But OK, I do also want to pause and say that so much of what I do has been built on the work of Margaret and other people in the community who have really, really stepped up and helped. And Bill, and so-- you know this is, our community is strong because of all of us together, not any one of us. So I've taken much more than I've given over the years, but that is kind of you to say. So what I realized though, ultimately, is maybe what I was sending the wrong message by just grading what they're turning in. I wanted to have them have an understanding of the metacognition, how are they learning this stuff. 

So what I do now is I say, OK once you've turned in the problem set, I want you to use the File Revision History to scroll back and find an aha moment where all of a sudden, you realize that something was-- that something was-- you'd been under a misapprehension about how something was working, or misunderstanding and you just fixed it, or talk about an aha moment. And they had to submit three of these moments along with their finished code. Which also makes it harder to just grab somebody else's code, because I want them to see, well, what was the story that you went through? And so you can't just say, well I moved this code up here and all of a sudden it worked. I want to see, how did you figure out how you fix something? 

And so here's an example of a story that kids put together when they turned in Caesar, like where were you stuck? How did you figure it out? So that's academic-- and I find if you only put the emphasis on the correct solution, then kids think, oh, what Mr. Kiang wants is the correct solution. So when they get crunched for time, they'll just be like, I'll just Google the correct solution and give it to him because that's what he wants. But that's not what I want. What I want is the learning, and I want to see evidence of learning. And I want you to struggle, and I want you to be stuck, and I want you to get yourself unstuck, because that's a most valuable part of CS50. It's not about the correct solution, I already have the correct solution. I have lots of correct solutions, I want to see your correct solution. So that's the message that I'm constantly giving kids. 

So the Create Task. So it is an AP course, a principles course. I do something a little bit different. I found in the past, trying to get kids to force fit a C program into what's being asked for by the College Board is difficult. So and sometimes I find that the more you know about programming, the harder it becomes to get a 5 on the exam, because kids kind of get themselves tripped up. The thing that really gets graded for the most part is the write up. So kids, for the AP course, they've got to basically come up with an independent project that uses a list, that uses a function, that takes a parameter, but then they have to write knowledgeably about it. And that's essentially where kids get tripped up. 

So I actually have kids do the Create Task first thing in the school year. So we're done with Create Task by August. We're done with it. We've already done it, we've written the write up, we don't do it in C we do it in Scratch. So when we start in Scratch for CS50, before we move on, we actually have them do their own independent project in Scratch and write about it. I have them practice a bunch of times for it, and I just get it out of the way and get it done. And then I don't even think about that portion of it again for the rest of the year. And this is not probably what the College Board would want you to do, because they want it to be a summative thing. But honestly the kids, in terms of their coding experience, I know if they go CS50 they're going to more than exceed what the College Board is looking for. Where they're going to get tripped up is in their explanation of it. 

So I have a link to a Scratch project here called MPolygon. And in this Scratch project, if I hit the play button, see how it just draws like five random polygons? It fills a list with random numbers of sides and then it draws them. And then if I hit Play again, it'll draw them again in random places, and sometimes it hits the edge and goes off a little bit. But that's it. So I teach kids how to do blocks. The first thing I teach them in Scratch is how to create a block, and then I say, OK, draw a square. And then create a block called square and move that code in there, and then create a block called square that takes a parameter for the length of the sides, and then draw a square of a variable side length. 

And within the first day, I mean within the first hour, they've already created a function that takes a parameter and that's one of the big requirements of the Create Task. And then it's just a matter of going over the list. But what I'll do is I'll show this to them, we go over the requirements of the Create Task I show this to them and then I say, OK you do the write-up and pretend you wrote this. Pretend you wrote this program, but now you do the write-up. And then I grade their write-ups. And then I have them go and do a project in Scratch just knowing these basics, a list, a parameter, a function. And it doesn't have to be huge. And then we get into C, and then we spend the rest of the year-- we don't even worry about the create task anymore. So that's one thing that I would do. 

The parameter is in when you're defining the polygon, so this pulls in a polygon, a number from the polygon list, and it actually changes. So if the number, if that parameter, is greater than 5, it sets the pen color to red. Otherwise it sets it to green. So this is not a great project, but this project, you could see scores on this from anywhere from 0 to 9 if you write it up-- if you write it up properly. And that's the whole thing. Everybody thinks the score on the Create Task is based on the programming, it's really not. It's based on the write-up. But anyway, this is what I've used. You could certainly create your own. In fact, I'd highly encourage you to create your own project that you think-- this is a minimal project, it meets the minimum requirements-- and then work on the write-up, that's the challenging thing. So yeah. I mean, the purpose of this would be entertainment to teach students about polygons having a certain number of sides or whatever, I mean it's all in the write-up. So anyway that's my take on the Create Task. Certainly take that with a grain of salt. 

But grating late work. So the problem with late work for me is that kids would tend to get stuck, and then not come in for help until the end of the quarter. And then they would have like five problem sets, and they want to meet with me to go over all five problem sets and it's just is not workable. And then if I grade kids-- if I give them a lower grade in my grade book, it's hard to tell in the grade book, if they got a 10 out of 15, is it because it was late? But it was actually a 15 out of 15, but it's just like five days late. Or was it truly flawed in some way, and that's why it's a 10 out of 15? So that ended up being kind of an issue. 

So I've come up with this tardies/late system with my partner teacher Eric at Punahou, and this worked-- it worked great, and we did this this year with Menlo and it worked great there too. So what we did is we differentiate between when something is tardy, and something is late, and here are our definitions. A tardy, so let's say the problems that was due Monday night. Tardy means if you don't get it turned in by Monday night, you automatically have a one day extension. You get it in turned in by Tuesday night, you're fine. And you don't have to ask me for permission, just turn it in by the following night. You don't need to provide me with an excuse. It's an automatic extension. They get three of those per semester, and they don't stack in the sense that you can't turn something in two nights late and say I'm going to use two tardies. It doesn't work, one per assignment. 

Now if they're are more than one day late, they can go into-- if they're more than-- if it comes in more than a day after the deadline, they have to use what's called a late. And late is a one week extension, it's seven days. And you can't just turn something in late, you have to meet with me. So you have to actually send me an invite, or say, hey, can I talk with you after class? Because a lot of the time when kids are late, if they can't figure it out they don't make the tardy deadline, and they don't come in for help. They just let it sit for a week. And now 6 days later, they're at the same place they were a week ago. And they're like, I still am stuck, and now I have to turn it in. So then they just turn in something that doesn't work. So in order for it to trigger a late, they have to come in and talk with me. And it has to be face to face, they can't just send me an email and say, I'm taking a late on this. And for those they get three per semester. 

And the way that I do this in Canvas is I keep track of it just a 3 point assignment, tardies and lates. So everybody starts with three tardies, three lates. They know the 3 tardies, each of them gives them a one day extension. Each of the lates gives them a one week extension. That covers all the problems sets you're going to do. And in some cases, you'll notice some kids actually have more than 3. That's because extra credit-- like if kids come to puzzle day, I usually will give them an extra tardy. So let's say, OK, great now you got four tardies. Kids often want extra credit, but not because they actually want it to-- a lot of the kids here really going for extra credit, the kids are already getting an A in the course anyway. They just want the recognition. So I'll get kids finishing the year with 4 tardies and 4 lates, and they feel proud of themselves. And it doesn't cost me anything, right? And the grade that's in the book, because I always grade the assignments when they come in, as I would grade them as if they were turned in on time. The grade in the book always reflects the quality of the assignment. And so that allows me to then calculate stuff based on all kinds of other stuff. 

So yeah, so finally for assessing projects, when you do an independent project-- and I would encourage you to do an independent project towards the end of the year that culminates in a share fair, where kids are sharing their work. But then how do you assess something like that? Well, what I do, is I think about, what is it that I want to reward, right? Or what is it that I want the grade to signify? Because if you consider the 3 profiles of kids, like let's say you have one student who meets all the intermediary deadlines, and they turn the project in on time. That student, of course, is probably going to get an A, right? You have student two, they're are a few days late on every check-in, and they turn in the final project two days late. That kid gets dinged on the project every single time they're late, so they probably get a B or a B-minus. Student three doesn't really meet any of the check-ins, but the last week of the project, they pull for all-nighters in a row and they turn the project in on time. That student usually gets an A, but that's not what I value. 

For me, I value consistent effort over time. And so, if you consider a student number one and number two, they actually did the same amount of work. But the issue with student two is they just had trouble turning stuff in on time, they need-- it needs to be late for them to realize they have to turn it in, right? But I'm actually happier with student one and student two than I am with student three. Student three already knew some C coming into the course, and they just figured out, oh, I can coast for all of this. And then I don't want to reward that, because if they had applied themselves consistently from the start, how much better could that project have been, right? Rather than just doing what you needed to do to clear the hurdle to get the A. 

So what I do is I emphasize the process. And I want them to document the process of their learning as they go, because in my mind the process is the story and the product is the ending. And we don't judge a story by its ending, right? Because like Romeo and Juliet, for example, it has a terrible ending, right? Everybody dies. I'm sorry if that's a spoiler for some people, but they do. Right? And that's horrible, right? People would say, well I would never want to read that. But the value in Romeo and Juliet is everything that happens up to the ending. The value in a project is everything that happens up to the time you turn it in. That's what I want them to document. 

And I think of it as a typical story structure, hero's journey, the call to action, you have the wisdom of the elder, the rising and falling action, I'm going to apply this to something we probably all know, or many of us. Star Wars, right? Or there are other stories like Star Wars, but it's the hero's journey, right? The call to action, a purpose bigger than yourself. Create an app that solves a real problem for real people. That's a call to action. As they go they have the wisdom of the elder, the Yoda. You as teachers, you're their Yoda, right? Yoda can't do the thing himself, Luke has to walk that path himself. Yoda can be there to give him advice, but ultimately it's Luke's independent project, he's got to do it, right? You are there to coach and to support, but you can't do the work for them. Only they can do that. 

Then there's this rising and falling action the thrills and chills of the movie. I mean that's where the bulk of the movie is, right? It's all the moments where you think everything is lost, and then all of a sudden this triumphant moment. That's what I want to hear, that's the story. Everybody thinks about oh, the Death Star blows up, that was the best part of Star Wars. Well no, it was all the points leading up to that when you weren't sure what was going to happen at the end. This is the payoff. The share fair is the payoff. And then the denouement, right? We have the celebration at the end, this is the share fair. 

This is the chance to sit back if you had group members. It's a chance to sit back and celebrate and talk about it. Oh my gosh, do you remember when we did this? Do you remember where we thought this was all lost, and then we figured out-- have kids relax and tell the stories. This is a healing time for a lot of group projects, because group projects are stressful. And you don't want any bad feelings that are created during the group project to go beyond. You want kids to realize that the classroom, the learning community, is bigger than any individual group project or group of people that they were working with. And that we value each other, each other's presence, and being with each other and supporting each other, over and above how any one project works. 

Now the specifics of this, and I've shared all this information and the shared drive as well about how I do this, but basically I engage them in personal goal setting. I conference with each group at least once during the project, I have them do work logs three times a week, and a record of thinking-- which is like a journal entry, and it culminates in a final narrative. And I'm going to talk with those last three pieces first. So then the work logs basically are just like how attorneys or accountants keep billable hours, all it is what I did, how long I spent doing it. That's it. It doesn't need to be more complicated than that. But I want them to keep track, as they work on the project, of what they're doing. Because everybody is working on different things. So I want to see what does that work-- what does that work log actually look like? 

And it shouldn't take them more than 30 seconds to do a work log. And if they were busy, or they had a soccer tournament and they couldn't get any work done, that goes in the workbook. I'll count that. But I want them, every other night, to be thinking about, where am I on this project? And then writing about it. And I don't take lates on the work logs, it's like they are on time or that's a 0. Because if you don't do that, then they save them all up till the end and then the value is kind of lost. A prompt that I might give them is I might say, when you do an independent project, and you propose one, I want you to show me what you know, so something old. 

Something new-- in other words go out and learn something on your own. Figure out how to do graphics, or figure out how to do this other thing, or how to do a form-- learn something new. Something borrowed, so I might point them over to GitHub and say, OK. Now it's OK to go look at, what are some other bits of code that people have? Other libraries people have made available to you? And so I've got the old, something new, something borrowed, and then, just because you have to, something blue. Well what does blue mean? I don't know. Could be some aspect of the interface. It could be I'm coloring the text blue. It could be-- one person did-- Eliza, they said they did a virtual psychologist, so that when you're feeling blue, when you're feeling depressed, it will kind of tell you, talk to you. So that's how they interpreted it, which I thought was creative. But that's the prompt. 

And then they go through and they've done all of this work, the record of thinking happens once a week, it's like a journal entry. And then the final narrative takes the place of the final exam at the end of the school year for me. What I have the kids do is they go through all of their record of thinking entries, their work logs, any notes that they took from meeting with me, and I want them to tell me the full story. And they have to cite their own work logs. So they can say, well, we struggled at first-- work log number 1-- until we figure it out how to do a dictionary in Python-- work log number 2-- they have to cite to it. And so at this point, if they haven't filled out the work logs leading up to it, they won't have anything to write for the final narrative. 

Because I don't want some revisionist history thing that they write at the end that says, well we struggled because it was hard at first, but we finally figured it out. Yay us! It's just pablum, right? I used to get that before I had kids actually, really go through, and give yourself credit for the areas where you struggled, and the areas where you figured things out. And I'll get things like this. Making something is-- we're frustrated together-- success and happiness. I mean most of the learning here involves them going back through and reading. Oh my gosh we really did work really hard on a lot of these different things. So that's where you see a lot of the value of it. 

And so that's kind of how I go about assessing these. And what I find is, every bit of opportunity that I give kids to tell me a little bit more about themselves helps me understand more about them, and helps me hear their voice and gives them a sense that they belong. And ultimately that's the most important part of creating a really positive supportive culture in the class. That's what I want them to take away from it. There are three things that I ask kids to repeat to themselves when they get stuck. Or we do this in class. I want them to say, I'm smart. I'm resourceful. I belong here. And I make them all repeat this as a group. I'm smart, I'm resourceful, I belong here. 

Because when they get stuck, there's a little voice in a lot of kids' head that tells them the opposite. I'm not smart enough to be here. I don't know what to do, I'm stuck. I don't belong here, I have no business taking a CS course. So I believe in positive affirmations, and I have them repeat that over and over again. And I tell them you do belong here. And I'm so glad that you're here, and I'm so glad that you're part of this class because this class would not be the same without you. Thank you for being here. And I tell them that over and over again. Because I think high school students, they need to hear that. All the time. Well, we need to hear that too, anybody. So with that said, that is-- that's what I have for you. Please do spend some time going back, going through some of the resources and taking a look at them. I do want to be respectful of your time. And I'll be around too for some of the reflections as well, if people have other follow up questions to ask. So thank you. Thanks again for the CS50 team, for putting together such a great series of tools and helping me be a better teacher. So thank you.