1 00:00:00,000 --> 00:00:02,240 2 00:00:02,240 --> 00:00:05,890 DAVID MALAN: All right welcome to the CS50x educator workshop. 3 00:00:05,890 --> 00:00:08,730 This is our session on grading programming assignments 4 00:00:08,730 --> 00:00:10,720 and handling academic honesty. 5 00:00:10,720 --> 00:00:13,720 We're joined today by CS50's own, Margaret Tanzoch, who's 6 00:00:13,720 --> 00:00:17,620 been teaching CS50 in her own classrooms for quite a few years 7 00:00:17,620 --> 00:00:19,870 and is here to share not only her experience, but also 8 00:00:19,870 --> 00:00:23,680 or advice on how to approach these challenges. 9 00:00:23,680 --> 00:00:26,535 Margaret, allow me to hand the floor to you. 10 00:00:26,535 --> 00:00:27,910 MARGARET TANZOCH: Hey, everybody. 11 00:00:27,910 --> 00:00:30,070 It's so nice to see you. 12 00:00:30,070 --> 00:00:33,040 As David mentioned-- thank you David, for that lovely introduction, 13 00:00:33,040 --> 00:00:36,670 I've been teaching 650 in high school for about five years. 14 00:00:36,670 --> 00:00:38,510 And what I'm going to go through today. 15 00:00:38,510 --> 00:00:40,690 I'm going to share with you what I do in terms 16 00:00:40,690 --> 00:00:43,090 of grading assignments and academic honesty. 17 00:00:43,090 --> 00:00:46,480 And I have to say, one of the things I love about the CS50 curriculum 18 00:00:46,480 --> 00:00:49,480 is that there's so much opportunity to kind of really 19 00:00:49,480 --> 00:00:53,230 run the classroom the way you want. 20 00:00:53,230 --> 00:00:55,373 There's no prescribed way that you have to do it. 21 00:00:55,373 --> 00:00:56,290 But let me just start. 22 00:00:56,290 --> 00:00:58,390 I'm going to share my screen. 23 00:00:58,390 --> 00:01:06,800 So to give you a little idea on my own background, in the high school I teach, 24 00:01:06,800 --> 00:01:11,270 called NEST+m New Explorations Into Technology, And Math, 25 00:01:11,270 --> 00:01:12,320 I have a lot of students. 26 00:01:12,320 --> 00:01:16,820 I teach generally about four sections of an intro computer science 27 00:01:16,820 --> 00:01:19,970 and one section of AP Computer Science principles. 28 00:01:19,970 --> 00:01:23,752 I have somewhere between 130 to 150 students. 29 00:01:23,752 --> 00:01:26,960 So again, the way I run my class might be a little different than if you just 30 00:01:26,960 --> 00:01:28,950 have a small group of students. 31 00:01:28,950 --> 00:01:31,910 I also teach CS50 at the Harvard Extension School 32 00:01:31,910 --> 00:01:33,740 and at Harvard Summer School. 33 00:01:33,740 --> 00:01:37,550 So I'm pretty familiar with a lot of the problem sets and the curriculum, 34 00:01:37,550 --> 00:01:38,760 as well. 35 00:01:38,760 --> 00:01:40,370 This is a picture of my high school. 36 00:01:40,370 --> 00:01:44,840 This was from a hackathon that we had last year, where David was kind enough 37 00:01:44,840 --> 00:01:46,520 to come and visit. 38 00:01:46,520 --> 00:01:48,290 We have, it's a K through 12 school. 39 00:01:48,290 --> 00:01:51,770 So although I usually teach juniors and seniors at my high school, 40 00:01:51,770 --> 00:01:53,660 we also invited middle schoolers. 41 00:01:53,660 --> 00:01:55,790 And everybody had an amazing time just using 42 00:01:55,790 --> 00:01:59,360 the day to have an extended time to really delve into some cool programming 43 00:01:59,360 --> 00:02:00,470 problems. 44 00:02:00,470 --> 00:02:02,870 So that's where I teach. 45 00:02:02,870 --> 00:02:05,310 So in terms of what we're going to talk about, 46 00:02:05,310 --> 00:02:08,630 hopefully, everybody had a chance pretty much to meet one of their neighbors. 47 00:02:08,630 --> 00:02:13,913 And by the way, I'm going to paste in, I think somebody's pasted it Google talk. 48 00:02:13,913 --> 00:02:15,080 I'm not sure if that's mine. 49 00:02:15,080 --> 00:02:17,210 I'll paste in the presentation that I'm doing, 50 00:02:17,210 --> 00:02:19,280 in case anybody wants to follow along. 51 00:02:19,280 --> 00:02:23,600 All right, so I'm going to talk a little bit about how I assign problems 52 00:02:23,600 --> 00:02:28,610 and what kind of deadlines I use, how I use the correctness and style scores, 53 00:02:28,610 --> 00:02:31,140 how I use design integrating process. 54 00:02:31,140 --> 00:02:33,890 And one little things that I do with my particular students, which 55 00:02:33,890 --> 00:02:35,150 are reflections. 56 00:02:35,150 --> 00:02:38,000 And then I'll talk a little bit about academic honesty 57 00:02:38,000 --> 00:02:42,020 and how I try to handle it, how I try to avoid 58 00:02:42,020 --> 00:02:44,360 that happening, and the culture that I create 59 00:02:44,360 --> 00:02:48,500 to kind of keep a lot of academic integrity in the classroom. 60 00:02:48,500 --> 00:02:51,680 And then there'll be plenty of opportunity for you to ask questions. 61 00:02:51,680 --> 00:02:55,320 I'm going to suggest that, as David had mentioned, if you do have a question, 62 00:02:55,320 --> 00:02:59,990 please raise your hands in the Raise hands option in the participants panel. 63 00:02:59,990 --> 00:03:04,140 And I'll pause and call at you at an appropriate time. 64 00:03:04,140 --> 00:03:08,220 All right, so starting are taking a look at grading programming assignments. 65 00:03:08,220 --> 00:03:12,150 Before we start, I want to talk a little bit about my own philosophy on grading. 66 00:03:12,150 --> 00:03:16,230 So in my high school, students take computer science as an elective. 67 00:03:16,230 --> 00:03:19,590 Many of them have no knowledge of what computer science is. 68 00:03:19,590 --> 00:03:21,510 Many of them are curious to get a sense of it 69 00:03:21,510 --> 00:03:24,593 to see if it's something they may want to major in in college or something 70 00:03:24,593 --> 00:03:25,998 they might have a talent for. 71 00:03:25,998 --> 00:03:29,040 Many of them feel a little bit nervous about coming into computer science 72 00:03:29,040 --> 00:03:32,760 because although they've used computers for so much of their life, what goes on 73 00:03:32,760 --> 00:03:35,790 inside that computer and how programs run is something 74 00:03:35,790 --> 00:03:39,390 that they have very, very little familiarity with. 75 00:03:39,390 --> 00:03:42,060 I don't really want students to feel afraid of taking my class 76 00:03:42,060 --> 00:03:43,980 because it's going to bring down their GPA. 77 00:03:43,980 --> 00:03:47,820 About 99 percent of the students at my school want to go on to college. 78 00:03:47,820 --> 00:03:50,940 And having a good GPA is very important to them. 79 00:03:50,940 --> 00:03:55,800 And so unlike in some colleges, students don't have the opportunity 80 00:03:55,800 --> 00:03:57,570 to take my class pass fail. 81 00:03:57,570 --> 00:03:59,910 So while I say this, I'm not a harsh grader. 82 00:03:59,910 --> 00:04:01,290 But the work is hard. 83 00:04:01,290 --> 00:04:04,980 I do give students a lot of opportunities to do well. 84 00:04:04,980 --> 00:04:06,840 I support them in their work. 85 00:04:06,840 --> 00:04:08,548 I allow them to collaborate. 86 00:04:08,548 --> 00:04:11,340 And I even offer like extra credit problems and writing assignments 87 00:04:11,340 --> 00:04:14,310 to try to even things out when students need a little bit of something 88 00:04:14,310 --> 00:04:17,790 extra to help out with their grade. 89 00:04:17,790 --> 00:04:20,709 Now I create a website for each of my classes. 90 00:04:20,709 --> 00:04:24,190 And I post weekly, pretty much, kind of the expectations 91 00:04:24,190 --> 00:04:25,840 for what we're working on that week. 92 00:04:25,840 --> 00:04:27,070 I post a program. 93 00:04:27,070 --> 00:04:30,700 Generally, one program a week, along with the program specs 94 00:04:30,700 --> 00:04:32,170 and tons of resources. 95 00:04:32,170 --> 00:04:35,710 So I'll post, for instance, the CS50 lecture. 96 00:04:35,710 --> 00:04:38,410 Even though I don't require that they watch the lecture, 97 00:04:38,410 --> 00:04:41,500 I will post it for them to have, if they choose to watch it. 98 00:04:41,500 --> 00:04:43,090 I will recommend that they watch it. 99 00:04:43,090 --> 00:04:45,980 I'll post many of the shorts that feature 100 00:04:45,980 --> 00:04:50,440 Doug Lloyd that delve in a little more detail into individual topics. 101 00:04:50,440 --> 00:04:52,660 I'll do a short mini lesson myself. 102 00:04:52,660 --> 00:04:54,820 And I'll give students most of the class time 103 00:04:54,820 --> 00:04:57,190 to work on their programming problems. 104 00:04:57,190 --> 00:05:00,970 I generally only assign writing problems for homework. 105 00:05:00,970 --> 00:05:05,050 Because again, some students, although the majority have computers 106 00:05:05,050 --> 00:05:08,030 and internet, I cannot assume that 100% of them do. 107 00:05:08,030 --> 00:05:09,867 So I want to keep things equitable. 108 00:05:09,867 --> 00:05:11,950 I don't want students to feel they have to stay up 109 00:05:11,950 --> 00:05:14,590 until 3:00 in the morning trying to debug a program, 110 00:05:14,590 --> 00:05:18,630 particularly they may have limited hours They could use their internet. 111 00:05:18,630 --> 00:05:21,130 They may have limited hours, they could use a home computer. 112 00:05:21,130 --> 00:05:24,870 So I give them lots of time to work on their problems in class. 113 00:05:24,870 --> 00:05:26,900 I'll allow them to work at their own pace. 114 00:05:26,900 --> 00:05:28,930 So I don't have harsh deadlines. 115 00:05:28,930 --> 00:05:30,850 I have suggested due dates. 116 00:05:30,850 --> 00:05:34,790 And students try very much to keep to those due dates, for the most part. 117 00:05:34,790 --> 00:05:37,930 But I do let students know that it's more important to me 118 00:05:37,930 --> 00:05:40,460 that they kind of work through the problem, 119 00:05:40,460 --> 00:05:42,490 that they understand what's going on with it. 120 00:05:42,490 --> 00:05:44,930 And maybe they do one less program or two less programs 121 00:05:44,930 --> 00:05:47,010 at the end of the marking period. 122 00:05:47,010 --> 00:05:50,800 Whereas I may give alternate assignment for some of the more difficult programs 123 00:05:50,800 --> 00:05:56,530 for students who need that than that try to kind of go crazy 124 00:05:56,530 --> 00:05:57,957 doing everything for homework. 125 00:05:57,957 --> 00:05:59,290 And again, we're in high school. 126 00:05:59,290 --> 00:06:03,130 So I try to keep it very reasonable amount of work. 127 00:06:03,130 --> 00:06:06,250 Students who finish early, and I do have some students that 128 00:06:06,250 --> 00:06:10,030 come into my class that already may have years of programming experience, 129 00:06:10,030 --> 00:06:12,760 or just really just fall in love with the material 130 00:06:12,760 --> 00:06:14,590 and end up working out the programs at home 131 00:06:14,590 --> 00:06:18,110 and working at a much faster pace than the typical student in the class. 132 00:06:18,110 --> 00:06:22,540 So for students who finish early, I allow them to become experts. 133 00:06:22,540 --> 00:06:25,960 I keep posters on the wall for each programming problem we're working on. 134 00:06:25,960 --> 00:06:29,710 And I allow those experts to help struggling students. 135 00:06:29,710 --> 00:06:33,310 In some schools, there are teaching Fellows, TFs or TAs, 136 00:06:33,310 --> 00:06:37,510 from some of the maybe past students or for students that have 137 00:06:37,510 --> 00:06:39,640 taken the course in previous years. 138 00:06:39,640 --> 00:06:41,560 We usually don't have an opportunity for that 139 00:06:41,560 --> 00:06:45,400 because my students are programmed pretty much for seven periods a day 140 00:06:45,400 --> 00:06:47,410 with only lunch as their break. 141 00:06:47,410 --> 00:06:49,300 So [INAUDIBLE] only one year ahead with TA. 142 00:06:49,300 --> 00:06:54,190 But in general, I depend on students in the classroom to help each other. 143 00:06:54,190 --> 00:06:55,668 I allow them to collaborate. 144 00:06:55,668 --> 00:06:57,460 And I will talk later about what that means 145 00:06:57,460 --> 00:07:00,310 to collaborate with academic integrity. 146 00:07:00,310 --> 00:07:03,160 So rather than have to have a program due and take points off 147 00:07:03,160 --> 00:07:05,290 if they can't get it in by the end of the week, 148 00:07:05,290 --> 00:07:08,630 I'll give students until the end of the unit to submit their programs. 149 00:07:08,630 --> 00:07:10,630 And for some students who are really struggling, 150 00:07:10,630 --> 00:07:13,360 who need some extra support, I'll even give them 151 00:07:13,360 --> 00:07:15,830 to the end of the marking period. 152 00:07:15,830 --> 00:07:18,340 So again, my students are a very diverse group. 153 00:07:18,340 --> 00:07:22,180 We're a considered a screen school, a gifted and talented school. 154 00:07:22,180 --> 00:07:25,340 Although we also have students with disabilities, 155 00:07:25,340 --> 00:07:27,730 students with special needs, IEP students. 156 00:07:27,730 --> 00:07:29,830 And the range of students in my classroom 157 00:07:29,830 --> 00:07:35,650 could be a student who is currently taking calculus as a senior, 158 00:07:35,650 --> 00:07:38,980 along with a student who failed algebra and is actually given to put 159 00:07:38,980 --> 00:07:40,660 in my class to get the math credit. 160 00:07:40,660 --> 00:07:43,215 So it's a very, very wide range of abilities, 161 00:07:43,215 --> 00:07:45,340 which is another reason why I really work carefully 162 00:07:45,340 --> 00:07:47,920 on creating a class culture where students 163 00:07:47,920 --> 00:07:51,280 understand that it's a process. 164 00:07:51,280 --> 00:07:53,920 It takes a while, sometimes, to get comfortable with program. 165 00:07:53,920 --> 00:07:55,780 And the mistakes are totally normal. 166 00:07:55,780 --> 00:07:56,960 It's part of the process. 167 00:07:56,960 --> 00:07:59,220 It's called debugging. 168 00:07:59,220 --> 00:08:03,550 I pose plenty of what CS50 calls some of the comfy versions of programs 169 00:08:03,550 --> 00:08:05,840 for extra credit for my advanced students. 170 00:08:05,840 --> 00:08:08,770 So my philosophy is that I never want anybody to get bored. 171 00:08:08,770 --> 00:08:11,740 There is always something for everybody to be doing at the level 172 00:08:11,740 --> 00:08:13,460 that you're able to handle it. 173 00:08:13,460 --> 00:08:16,330 And I do allow students to resubmit their programs as many times 174 00:08:16,330 --> 00:08:20,200 as they want to, up until pretty much the end of the say, 175 00:08:20,200 --> 00:08:24,310 for instance, the end of the unit, so that they can improve their grade. 176 00:08:24,310 --> 00:08:29,500 And the way that the tools work really make it very possible to do that. 177 00:08:29,500 --> 00:08:31,780 So I'll have my students submit all their programs, 178 00:08:31,780 --> 00:08:33,610 of course, using Submit 50. 179 00:08:33,610 --> 00:08:36,669 Submit 50, as you've seen in previous sessions this week, 180 00:08:36,669 --> 00:08:40,120 assign a corrected the score based on the Check50 for that problem. 181 00:08:40,120 --> 00:08:43,750 And also the style score based on indentation and spacing. 182 00:08:43,750 --> 00:08:48,070 Now you create whatever formula you want to create and get a grade for this. 183 00:08:48,070 --> 00:08:51,610 What I do is I use 75% of the correctness score 184 00:08:51,610 --> 00:08:54,190 and 25% of the style score. 185 00:08:54,190 --> 00:08:55,990 And that's how I assign grades. 186 00:08:55,990 --> 00:08:59,050 I actually automate it a little further by downloading, 187 00:08:59,050 --> 00:09:02,770 from these Submit50 Data [INAUDIBLE] platform, I'll download a CSV file 188 00:09:02,770 --> 00:09:04,930 and I'll run a little Python script to be 189 00:09:04,930 --> 00:09:07,540 able to calculate grades using this formula. 190 00:09:07,540 --> 00:09:11,230 And then I'll map students GitHub names to their student ID numbers 191 00:09:11,230 --> 00:09:14,530 and create a CSV file that I can upload to the school grade book. 192 00:09:14,530 --> 00:09:18,530 So that makes it a little easier for me to read grade programs, as well. 193 00:09:18,530 --> 00:09:21,580 The other thing that I do is that as I'm going through this, 194 00:09:21,580 --> 00:09:25,000 I'll take a look at some scores might sometimes 195 00:09:25,000 --> 00:09:27,370 look like a very low correctness score. 196 00:09:27,370 --> 00:09:30,010 And in that case, I do make sure to look at the code 197 00:09:30,010 --> 00:09:32,740 and tweak the score if the error was minor. 198 00:09:32,740 --> 00:09:35,680 Because sometimes, students will, at the last minute, 199 00:09:35,680 --> 00:09:39,430 try to fix the style on their code, even though the program is running 200 00:09:39,430 --> 00:09:41,950 and they seem to be passing all the correctness checks. 201 00:09:41,950 --> 00:09:44,350 But in the process of fixing the style, they 202 00:09:44,350 --> 00:09:48,610 might accidentally delete one of the forward slashes for a comment 203 00:09:48,610 --> 00:09:50,980 or possibly introduce a straight character 204 00:09:50,980 --> 00:09:55,420 into their code, which might cause the compilation to fail when the Check 205 00:09:55,420 --> 00:09:57,880 50 is run during the Submit 50 process. 206 00:09:57,880 --> 00:10:00,880 So I will take a look at that because I knew what the grades to be fair. 207 00:10:00,880 --> 00:10:03,755 I'll usually mention it to the student if I could catch them and have 208 00:10:03,755 --> 00:10:04,970 them correct themselves. 209 00:10:04,970 --> 00:10:07,600 But if it's at the very last moment, there's no time, 210 00:10:07,600 --> 00:10:09,310 I will take that into account. 211 00:10:09,310 --> 00:10:12,670 And I'll adjust the grade on that. 212 00:10:12,670 --> 00:10:13,680 So this is an example. 213 00:10:13,680 --> 00:10:16,650 This is actually a real student that submitted Cash 214 00:10:16,650 --> 00:10:19,680 at the beginning of the school year last year. 215 00:10:19,680 --> 00:10:21,960 As you can see, the student submitted twice. 216 00:10:21,960 --> 00:10:25,110 And the first time they only got 4 out of 11 checks 217 00:10:25,110 --> 00:10:29,700 correctly and 74% of the style done correctly. 218 00:10:29,700 --> 00:10:33,990 And so a couple of weeks later, they were able to resubmit their program. 219 00:10:33,990 --> 00:10:37,500 They got a better Check 50 grade and a better style grade. 220 00:10:37,500 --> 00:10:40,830 And so the way that I assigned the grade for this student 221 00:10:40,830 --> 00:10:48,330 was to multiply that 10 over 11 by 75 plus the 90 times the 25 percent. 222 00:10:48,330 --> 00:10:53,760 So this student earned a 90.7 percent on this particular program. 223 00:10:53,760 --> 00:10:56,130 And I'll always use the highest score. 224 00:10:56,130 --> 00:10:59,190 So sometimes, the student may try to correct something. 225 00:10:59,190 --> 00:11:02,730 And occasionally, as they corrected, they may miss something else up. 226 00:11:02,730 --> 00:11:06,060 So I do have my Python script make sure to check it. 227 00:11:06,060 --> 00:11:09,195 And it'll only pick up the highest score when I'm creating the grades. 228 00:11:09,195 --> 00:11:12,590 229 00:11:12,590 --> 00:11:17,630 Here's an example of the student, as well, that submitted Cash perfectly 230 00:11:17,630 --> 00:11:18,470 correctly. 231 00:11:18,470 --> 00:11:22,250 So as you can see, the check 50, since there were 11 checks for this program, 232 00:11:22,250 --> 00:11:25,400 the score came out to be 11 over 11. 233 00:11:25,400 --> 00:11:28,560 And the style came out to be 100%. 234 00:11:28,560 --> 00:11:31,720 And so the student earned 100% on this particular program. 235 00:11:31,720 --> 00:11:34,400 236 00:11:34,400 --> 00:11:38,200 Now in terms of design, some teachers are very big on design. 237 00:11:38,200 --> 00:11:41,810 For my introductory students, for almost all of them, 238 00:11:41,810 --> 00:11:44,170 this is the first time they're doing any coding at all. 239 00:11:44,170 --> 00:11:46,510 I'll strictly-- and I also have a lot of them. 240 00:11:46,510 --> 00:11:49,720 I might have 100 or more introductory students. 241 00:11:49,720 --> 00:11:52,030 I'll use the correctness and the style score 242 00:11:52,030 --> 00:11:54,730 as a grade for that particular program. 243 00:11:54,730 --> 00:11:58,210 For my AP students, I will sometimes require additional criteria, 244 00:11:58,210 --> 00:11:59,920 depending on what the problem is. 245 00:11:59,920 --> 00:12:03,940 So for instance, when they are creating a program like Caesar or Vigenére, 246 00:12:03,940 --> 00:12:08,170 I do enforce that they use a function because functions are really important. 247 00:12:08,170 --> 00:12:11,290 A lot of my students do seem to struggle with functions. 248 00:12:11,290 --> 00:12:15,040 And so I will not give 100% of the program if they don't use a function. 249 00:12:15,040 --> 00:12:19,480 I'll make sure that they avoid any type of repetitive or unnecessary code, 250 00:12:19,480 --> 00:12:21,535 make sure they use good variable names. 251 00:12:21,535 --> 00:12:24,160 There always seems to be a student that wants to use like, Bob, 252 00:12:24,160 --> 00:12:25,490 as a variable name. 253 00:12:25,490 --> 00:12:27,620 So we'll have a discussion about that. 254 00:12:27,620 --> 00:12:31,930 And again, of course, I'll enforce that they use reasonable variable names 255 00:12:31,930 --> 00:12:33,490 and also comment their code. 256 00:12:33,490 --> 00:12:37,470 257 00:12:37,470 --> 00:12:42,180 I do design feedback by looking at the students submissions 258 00:12:42,180 --> 00:12:47,220 and finding common less than optimal design 259 00:12:47,220 --> 00:12:49,660 choices that multiple students make. 260 00:12:49,660 --> 00:12:54,930 So for instance, from Cash this is the way that many students in my class 261 00:12:54,930 --> 00:12:59,850 have created the algorithm for the CASH problem set, right? 262 00:12:59,850 --> 00:13:01,740 So the idea here is you're trying to-- 263 00:13:01,740 --> 00:13:04,500 Cash is basically a greedy algorithm that's 264 00:13:04,500 --> 00:13:07,200 asking you to figure out the minimum number of coins 265 00:13:07,200 --> 00:13:10,260 that you can get from a certain number of change. 266 00:13:10,260 --> 00:13:13,000 So just to kind of open it up, what do you guys think? 267 00:13:13,000 --> 00:13:15,940 How could this design be improved? 268 00:13:15,940 --> 00:13:19,180 Justin, I see your hand's up would you like to respond to that? 269 00:13:19,180 --> 00:13:22,510 AUDIENCE: OK, I had raised my hand to ask a question. 270 00:13:22,510 --> 00:13:28,930 But I'm not prepped for this So sorry, I'll pass. 271 00:13:28,930 --> 00:13:31,180 MARGARET TANZOCH: Would you like to ask your question? 272 00:13:31,180 --> 00:13:33,210 AUDIENCE: Oh, sure, if that's appropriate. 273 00:13:33,210 --> 00:13:35,110 I would like to know-- 274 00:13:35,110 --> 00:13:40,600 because I heard you talk about how you manage the different level students 275 00:13:40,600 --> 00:13:42,080 and it sounds like a good strategy. 276 00:13:42,080 --> 00:13:47,990 And I heard David before mention to encourage students to just think 277 00:13:47,990 --> 00:13:54,060 about their progress when you start the class versus how to end it and so on. 278 00:13:54,060 --> 00:13:57,980 But there is a particular area that we struggle with here 279 00:13:57,980 --> 00:13:59,900 as a class discussion. 280 00:13:59,900 --> 00:14:04,520 And the students that are advanced tend to dominate 281 00:14:04,520 --> 00:14:08,720 because they are passionate about what they know and want to share it. 282 00:14:08,720 --> 00:14:13,180 And of course, there were other studies where like for example women, 283 00:14:13,180 --> 00:14:18,620 this particular about math, specifically, will do worse 284 00:14:18,620 --> 00:14:21,560 if men are just in the room. 285 00:14:21,560 --> 00:14:25,820 Or Malcolm Gladwell research that if you're a top student, 286 00:14:25,820 --> 00:14:29,840 you get more attention from the teacher and you do better. 287 00:14:29,840 --> 00:14:33,980 How do you manage that live discussion, the those students don't 288 00:14:33,980 --> 00:14:37,220 intimidate average and other students and everyone 289 00:14:37,220 --> 00:14:40,068 can work on their own progress? 290 00:14:40,068 --> 00:14:42,110 MARGARET TANZOCH: Well, I'll do different things. 291 00:14:42,110 --> 00:14:44,210 My students are organized in my classroom, 292 00:14:44,210 --> 00:14:47,545 at least before we work remotely, at tables. 293 00:14:47,545 --> 00:14:50,420 And so I'll often have students turn and talk with their small groups 294 00:14:50,420 --> 00:14:51,590 about a problem. 295 00:14:51,590 --> 00:14:53,540 And then I'll ask somebody from each table 296 00:14:53,540 --> 00:14:55,860 to share with that group talked about. 297 00:14:55,860 --> 00:14:59,090 And so in that way, hopefully, everybody has something 298 00:14:59,090 --> 00:15:00,340 that they can offer up, right? 299 00:15:00,340 --> 00:15:03,590 So even if a student didn't feel confident up front 300 00:15:03,590 --> 00:15:07,790 talking about maybe a design choice on a problem like this, in many cases, 301 00:15:07,790 --> 00:15:10,830 they got some information from somebody else at the table. 302 00:15:10,830 --> 00:15:14,900 And so they may feel a little bit more comfortable speaking up. 303 00:15:14,900 --> 00:15:18,470 The other thing is that I do fairly short lessons 304 00:15:18,470 --> 00:15:21,540 on a very small topic on a daily basis. 305 00:15:21,540 --> 00:15:23,930 So I might just do a little lesson on a for loop. 306 00:15:23,930 --> 00:15:27,410 And then I'll spend have the students spent most of their time doing 307 00:15:27,410 --> 00:15:28,970 coding during that period. 308 00:15:28,970 --> 00:15:30,360 And I'll walk around the room. 309 00:15:30,360 --> 00:15:33,578 I'll encourage them to work, sometimes, together, maybe 310 00:15:33,578 --> 00:15:35,870 have small groups sit together and psuedocode a problem 311 00:15:35,870 --> 00:15:38,420 before they start working on it. 312 00:15:38,420 --> 00:15:40,160 I'll talk to them individually. 313 00:15:40,160 --> 00:15:42,563 314 00:15:42,563 --> 00:15:44,230 Yeah, I mean, that's an issue sometimes. 315 00:15:44,230 --> 00:15:46,800 I mean, there are some students that are very shy and quiet 316 00:15:46,800 --> 00:15:48,850 and just don't have a lot of confidence. 317 00:15:48,850 --> 00:15:53,400 And so I'll try to work with them and build up their confidence and really 318 00:15:53,400 --> 00:15:55,920 kind of stress how they could see how far they've 319 00:15:55,920 --> 00:15:58,530 come during the course of a few months in the class 320 00:15:58,530 --> 00:16:02,100 and certainly by the end of the first semester. 321 00:16:02,100 --> 00:16:07,200 I love to do-- there's an AP-- a cs50 AP program writing problem, actually, 322 00:16:07,200 --> 00:16:11,100 called, Analyze This, which I always do at the end of semester 1, 323 00:16:11,100 --> 00:16:15,930 where students are asked about their growth as a computer scientist. 324 00:16:15,930 --> 00:16:20,850 And I love to assign that and really encourage students to see 325 00:16:20,850 --> 00:16:24,900 for themselves how much they've learned during that particular semester, 326 00:16:24,900 --> 00:16:29,460 particularly students that often have no understanding of what coding even is 327 00:16:29,460 --> 00:16:33,630 to being able to complete programs like some of the cryptography programs, 328 00:16:33,630 --> 00:16:35,460 Caesar, Vigenére, and beyond. 329 00:16:35,460 --> 00:16:37,615 So it is a work in progress. 330 00:16:37,615 --> 00:16:39,240 I'm not going to say I have it perfect. 331 00:16:39,240 --> 00:16:43,620 But I do try to encourage a small group discussion. 332 00:16:43,620 --> 00:16:47,430 I'll also pull together small groups of students to sit around a table, 333 00:16:47,430 --> 00:16:50,747 if they're not quite competent, and to help them with their program, 334 00:16:50,747 --> 00:16:52,830 so that they feel a little bit confident, as well, 335 00:16:52,830 --> 00:16:56,700 in terms of understanding what the steps to solve a problem might be. 336 00:16:56,700 --> 00:17:01,240 So I hope that I answered your question. 337 00:17:01,240 --> 00:17:03,100 AUDIENCE: Another, sorry Margaret. 338 00:17:03,100 --> 00:17:06,310 Another question that came up in the chat from David [INAUDIBLE],, 339 00:17:06,310 --> 00:17:08,998 is what is the average age of your students? 340 00:17:08,998 --> 00:17:12,040 MARGARET TANZOCH: My students tend to be juniors and seniors so they tend 341 00:17:12,040 --> 00:17:15,069 to be 16 and 17, for the most part. 342 00:17:15,069 --> 00:17:20,079 I have occasionally had one ninth grade student, who was 14 and [INAUDIBLE].. 343 00:17:20,079 --> 00:17:24,220 But mostly 16, 17, 18 something in that range. 344 00:17:24,220 --> 00:17:26,520 Good question, thank you. 345 00:17:26,520 --> 00:17:29,540 Other questions. 346 00:17:29,540 --> 00:17:31,720 I see a number of hands raised. 347 00:17:31,720 --> 00:17:33,792 [INAUDIBLE], do you have a question? 348 00:17:33,792 --> 00:17:34,500 AUDIENCE: Oh, no. 349 00:17:34,500 --> 00:17:38,400 I just to answer the question how the design can be improved. 350 00:17:38,400 --> 00:17:42,000 We can use some mathematics. 351 00:17:42,000 --> 00:17:45,840 Yeah, we can use integer division with remainder. 352 00:17:45,840 --> 00:17:48,690 And then it's very simple. 353 00:17:48,690 --> 00:17:52,680 We just divide by 25 [INAUDIBLE] remainder. 354 00:17:52,680 --> 00:17:58,890 And the integer quotient is the number of coins we will get. 355 00:17:58,890 --> 00:18:03,580 And then we pass to the next one, the remainder. 356 00:18:03,580 --> 00:18:06,000 So it will be much faster. 357 00:18:06,000 --> 00:18:10,500 And I implement it actually both versions with this while loop. 358 00:18:10,500 --> 00:18:14,670 And with integer remainder, integer remainder is so much faster. 359 00:18:14,670 --> 00:18:17,160 MARGARET TANZOCH: Excellent. that's excellent, great. 360 00:18:17,160 --> 00:18:17,850 What about this? 361 00:18:17,850 --> 00:18:21,300 If we even just looked at this code, if indeed we said, well, 362 00:18:21,300 --> 00:18:23,260 while loops are it acceptable approach. 363 00:18:23,260 --> 00:18:24,990 This is a new group of students, right? 364 00:18:24,990 --> 00:18:27,990 So what else if we were to use while loops might 365 00:18:27,990 --> 00:18:29,640 be something we could improve here? 366 00:18:29,640 --> 00:18:33,318 Anybody have any thoughts on that? 367 00:18:33,318 --> 00:18:35,360 [INAUDIBLE],, are you raising your hand for this? 368 00:18:35,360 --> 00:18:38,620 Or did you have a different question? 369 00:18:38,620 --> 00:18:40,585 AUDIENCE: [INAUDIBLE] here from India. 370 00:18:40,585 --> 00:18:43,460 MARGARET TANZOCH: Did you want to respond to the design choices here? 371 00:18:43,460 --> 00:18:45,130 AUDIENCE: Yeah, a thought [INAUDIBLE]. 372 00:18:45,130 --> 00:18:48,590 We can have a one way loop that we can have a [INAUDIBLE] greater than 373 00:18:48,590 --> 00:18:49,670 or equal to 1. 374 00:18:49,670 --> 00:18:52,410 And all this can be placed in a speed statement. 375 00:18:52,410 --> 00:18:55,130 There I can say the percent is 25. 376 00:18:55,130 --> 00:18:57,590 Then we can have cents equal to cents minus 45. 377 00:18:57,590 --> 00:18:59,338 Similar for 10, 5, and 1. 378 00:18:59,338 --> 00:19:01,130 And I would say this [INAUDIBLE] statement, 379 00:19:01,130 --> 00:19:03,070 we can have clients equal to plus 1. 380 00:19:03,070 --> 00:19:05,550 Because it is common for all that while loop. 381 00:19:05,550 --> 00:19:09,230 So one while loop and one switch statement can work for this. 382 00:19:09,230 --> 00:19:11,940 MARGARET TANZOCH: OK, all right, thank you. 383 00:19:11,940 --> 00:19:14,730 Anybody else have anything to add? 384 00:19:14,730 --> 00:19:15,892 Is that [INAUDIBLE]? 385 00:19:15,892 --> 00:19:17,850 I'm not sure if I'm pronouncing that correctly. 386 00:19:17,850 --> 00:19:18,910 I apologize if I'm not. 387 00:19:18,910 --> 00:19:22,150 388 00:19:22,150 --> 00:19:25,450 AUDIENCE: [INAUDIBLE] is my name, from Germany. 389 00:19:25,450 --> 00:19:30,650 Now I'm thinking have you talked about [INAUDIBLE].. 390 00:19:30,650 --> 00:19:40,370 So instead of saying greater than 25 and then cents minus 25, like x and an x, 391 00:19:40,370 --> 00:19:41,430 in both cases. 392 00:19:41,430 --> 00:19:45,800 So it doesn't matter what number is in there. 393 00:19:45,800 --> 00:19:51,230 And ask for input of the number that is used in this while statement. 394 00:19:51,230 --> 00:19:53,628 395 00:19:53,628 --> 00:19:56,170 MARGARET TANZOCH: I'm not 100% sure I understand your comment 396 00:19:56,170 --> 00:19:59,650 that instead of the variable names cents, we would use-- 397 00:19:59,650 --> 00:20:04,030 AUDIENCE: No, instead of the 25, which is a fixed 398 00:20:04,030 --> 00:20:07,990 number which can be used only once. 399 00:20:07,990 --> 00:20:12,580 Instead of the 25, put the variable in and fill that 400 00:20:12,580 --> 00:20:17,480 with an input that is taken from the user. 401 00:20:17,480 --> 00:20:23,187 And then you can use it the while loop for every number that is given in. 402 00:20:23,187 --> 00:20:24,520 MARGARET TANZOCH: Got it, right. 403 00:20:24,520 --> 00:20:25,390 No, absolutely. 404 00:20:25,390 --> 00:20:28,120 That's another great approach. 405 00:20:28,120 --> 00:20:30,760 What about this while cents is greater than or equal to 1. 406 00:20:30,760 --> 00:20:33,330 Anybody have any thoughts about that part? 407 00:20:33,330 --> 00:20:39,040 AUDIENCE: Everybody, I just see repetitive tasks which 408 00:20:39,040 --> 00:20:44,290 can be used to the full function here. 409 00:20:44,290 --> 00:20:48,130 We can just create a function and substitute 410 00:20:48,130 --> 00:20:54,480 the number, which we should have substriked to do the function variable. 411 00:20:54,480 --> 00:20:57,910 MARGARET TANZOCH: OK, Ignatio? 412 00:20:57,910 --> 00:21:03,290 AUDIENCE: Well, I was talking about was functioning too 413 00:21:03,290 --> 00:21:07,860 but just about this sense equals 1. 414 00:21:07,860 --> 00:21:12,800 At the [INAUDIBLE] you just have the number of coins of 1 cents. 415 00:21:12,800 --> 00:21:15,212 So you'll not be in this loop. 416 00:21:15,212 --> 00:21:17,420 MARGARET TANZOCH: Exactly, this while loop at the end 417 00:21:17,420 --> 00:21:19,370 really is not necessary at all, right? 418 00:21:19,370 --> 00:21:23,877 Because as Ignatio said, that whatever number of cents is left 419 00:21:23,877 --> 00:21:26,210 is going to be the number of coins that are left, right? 420 00:21:26,210 --> 00:21:28,880 Because after you've taken out everything up to your nickels, 421 00:21:28,880 --> 00:21:30,860 the only thing you have left are pennies. 422 00:21:30,860 --> 00:21:33,890 So many different ways for improvement, maybe even 423 00:21:33,890 --> 00:21:39,620 using magic numbers instead of hard coding these values 25, 10, 5, and 1. 424 00:21:39,620 --> 00:21:41,970 So many different things to discuss about this. 425 00:21:41,970 --> 00:21:44,930 And so I will encourage those design choices 426 00:21:44,930 --> 00:21:49,010 to be more of a class discussion than enforcing that in the grading 427 00:21:49,010 --> 00:21:52,740 policy, for the most part. 428 00:21:52,740 --> 00:21:55,700 I also may have quizzes during the course of the year that 429 00:21:55,700 --> 00:21:58,020 ask students to improve on the design. 430 00:21:58,020 --> 00:22:03,020 So those are the ways that I will discuss, have those design discussions. 431 00:22:03,020 --> 00:22:05,900 Thank you for all of your suggestions. 432 00:22:05,900 --> 00:22:10,160 Another thing I do is that I have students do reflections 433 00:22:10,160 --> 00:22:12,080 after each programming problem. 434 00:22:12,080 --> 00:22:15,830 And I find this, for my class, to be a really important part 435 00:22:15,830 --> 00:22:19,560 of my own assessment in terms of what a student really understands. 436 00:22:19,560 --> 00:22:24,500 So a reflection would necessarily discuss their development process, 437 00:22:24,500 --> 00:22:27,410 perhaps an aha moment or a struggle that they've encountered 438 00:22:27,410 --> 00:22:29,270 while they were creating their program. 439 00:22:29,270 --> 00:22:31,130 I'll usually have a very specific question 440 00:22:31,130 --> 00:22:35,000 that asks them to explain a specific algorithm in that program. 441 00:22:35,000 --> 00:22:39,508 And in some cases, perhaps have them collaborate. 442 00:22:39,508 --> 00:22:41,300 If they were collaborating on that program, 443 00:22:41,300 --> 00:22:44,600 have them discuss that particular collaboration. 444 00:22:44,600 --> 00:22:47,980 And this really also helps reveal to me if some students 445 00:22:47,980 --> 00:22:50,430 maybe collaborated a little too much, maybe 446 00:22:50,430 --> 00:22:52,250 doesn't have a real understanding. 447 00:22:52,250 --> 00:22:55,647 And allows me to kind of zoom in and then have more clear discussion 448 00:22:55,647 --> 00:22:56,855 with that particular student. 449 00:22:56,855 --> 00:22:59,740 450 00:22:59,740 --> 00:23:02,570 Semester grades, I use a combination of things. 451 00:23:02,570 --> 00:23:05,350 We use a grade book called IO classroom. 452 00:23:05,350 --> 00:23:07,000 It used to be called Schedula. 453 00:23:07,000 --> 00:23:10,030 And we have various different categories that are weighted. 454 00:23:10,030 --> 00:23:12,070 So for instance, the scores on the programming 455 00:23:12,070 --> 00:23:17,530 problems from the check50 and the style50 are one component of it. 456 00:23:17,530 --> 00:23:20,510 I'll also assign writing programs throughout the year. 457 00:23:20,510 --> 00:23:22,330 And those are the types of problems that I 458 00:23:22,330 --> 00:23:24,330 would tend to give for homework because it's not 459 00:23:24,330 --> 00:23:28,030 a problem that students need to struggle debugging in the middle of the night. 460 00:23:28,030 --> 00:23:30,430 And the writing programs, the cs50 AP curriculum 461 00:23:30,430 --> 00:23:32,630 does have some writing programs. 462 00:23:32,630 --> 00:23:36,220 And sometimes, I will have students write a short essay 463 00:23:36,220 --> 00:23:38,080 on some type of occurred event. 464 00:23:38,080 --> 00:23:41,740 So sometimes, I'll even have these essays substitute 465 00:23:41,740 --> 00:23:46,060 for maybe a more difficult problem that a student doesn't 466 00:23:46,060 --> 00:23:48,950 feel competent in solving, as an alternate assessment. 467 00:23:48,950 --> 00:23:53,920 So for instance, this year with the coronavirus as such a major topic 468 00:23:53,920 --> 00:23:58,600 in the news, I had students write a short essay on computer simulation 469 00:23:58,600 --> 00:24:02,710 and modeling and used a little bit of a discussion about the virus 470 00:24:02,710 --> 00:24:06,710 and how it replicates to kind of introduce that particular topic. 471 00:24:06,710 --> 00:24:09,250 So I like the idea of writing assignments 472 00:24:09,250 --> 00:24:13,100 to also bring what we're doing in class into a real world environment. 473 00:24:13,100 --> 00:24:15,280 And that allows us to have more timely discussions 474 00:24:15,280 --> 00:24:17,530 on various different topics. 475 00:24:17,530 --> 00:24:19,210 Also grade on classwork. 476 00:24:19,210 --> 00:24:23,320 And that would be partly involved in class discussion, how focused, 477 00:24:23,320 --> 00:24:25,120 engaged students are. 478 00:24:25,120 --> 00:24:27,880 And also allowing and knowing that some students are really shy 479 00:24:27,880 --> 00:24:29,130 you don't have the confidence. 480 00:24:29,130 --> 00:24:31,270 So I take them to account, as well. 481 00:24:31,270 --> 00:24:32,770 I do do quizzes throughout the year. 482 00:24:32,770 --> 00:24:35,080 And I'll be talking a little more about that tomorrow. 483 00:24:35,080 --> 00:24:36,997 And then if reflections, as I mentioned, which 484 00:24:36,997 --> 00:24:40,270 I find to be a really important part of students 485 00:24:40,270 --> 00:24:42,940 reflecting on their own program development 486 00:24:42,940 --> 00:24:47,200 and also informing me on what the development looks like. 487 00:24:47,200 --> 00:24:50,030 Also, it just helps me know were the problems too easy or too hard 488 00:24:50,030 --> 00:24:52,923 for the students, kind of where we are in assessing that. 489 00:24:52,923 --> 00:24:54,590 So I'd like to take a little break here. 490 00:24:54,590 --> 00:24:58,410 And let's take some more questions. 491 00:24:58,410 --> 00:25:00,260 Ahmad, do you have a question? 492 00:25:00,260 --> 00:25:02,030 AUDIENCE: Yes, Margaret. 493 00:25:02,030 --> 00:25:03,810 Thank you. 494 00:25:03,810 --> 00:25:13,580 I wanted to ask you about something regarding the grading or the comments, 495 00:25:13,580 --> 00:25:14,780 which is peer review. 496 00:25:14,780 --> 00:25:16,573 Have you tried? 497 00:25:16,573 --> 00:25:19,490 I remember that you said that you teach also at Harvard Summer School, 498 00:25:19,490 --> 00:25:21,620 maybe at the university level. 499 00:25:21,620 --> 00:25:25,100 Have you tried giving students the opportunity 500 00:25:25,100 --> 00:25:33,375 to peer review solutions from others students to maybe grading them? 501 00:25:33,375 --> 00:25:35,750 MARGARET TANZOCH: I think that's an excellent suggestion. 502 00:25:35,750 --> 00:25:36,875 I've done a little of that. 503 00:25:36,875 --> 00:25:39,080 I would like to do more of that. 504 00:25:39,080 --> 00:25:41,820 One of the reasons I haven't done that as much as I'd like 505 00:25:41,820 --> 00:25:44,990 is that students tend to finish the problems at different times 506 00:25:44,990 --> 00:25:48,630 because I have such a wide assortment of abilities in the classroom. 507 00:25:48,630 --> 00:25:51,320 And so some students will finish a program a day. 508 00:25:51,320 --> 00:25:54,890 Another student might take two weeks and really be struggling through it 509 00:25:54,890 --> 00:25:55,940 and working on that. 510 00:25:55,940 --> 00:25:58,010 So I haven't-- 511 00:25:58,010 --> 00:25:59,300 I do that occasionally. 512 00:25:59,300 --> 00:26:02,245 I have students, certainly, at the end of the year when 513 00:26:02,245 --> 00:26:04,370 I have them do final projects, I have them write up 514 00:26:04,370 --> 00:26:06,680 a proposal that they review each other's proposals. 515 00:26:06,680 --> 00:26:10,790 And they give each other suggestions, how to make it better, 516 00:26:10,790 --> 00:26:12,110 how to make it clearer. 517 00:26:12,110 --> 00:26:14,810 And I find that's a really great process to go through. 518 00:26:14,810 --> 00:26:18,830 I do have students sometimes work together on their problems, 519 00:26:18,830 --> 00:26:21,080 particularly if they're working at the same level. 520 00:26:21,080 --> 00:26:24,530 So I may have two students that are struggling a little bit. 521 00:26:24,530 --> 00:26:26,087 And I'll allow them to both develop. 522 00:26:26,087 --> 00:26:27,920 So their code will look the same at the end, 523 00:26:27,920 --> 00:26:29,930 but I know they've worked it through together. 524 00:26:29,930 --> 00:26:31,470 And so I'm OK with that. 525 00:26:31,470 --> 00:26:34,340 So they will kind of go back and forth. 526 00:26:34,340 --> 00:26:36,090 I'll see the two of them working together. 527 00:26:36,090 --> 00:26:38,810 And that can be really productive, as well. 528 00:26:38,810 --> 00:26:40,760 And then they will discuss each other's code. 529 00:26:40,760 --> 00:26:44,630 So I think there's a lot of ideas that could be implemented, depending 530 00:26:44,630 --> 00:26:46,280 on the profile of your classroom. 531 00:26:46,280 --> 00:26:49,490 So but that's a great question. 532 00:26:49,490 --> 00:26:53,740 [INAUDIBLE] I'm sorry if I mispronounced that. 533 00:26:53,740 --> 00:26:56,860 Do you have a question? 534 00:26:56,860 --> 00:26:59,330 AUDIENCE: Thank you, Professor Margaret. 535 00:26:59,330 --> 00:27:04,993 To shortly, the first one, do practice to share the best projects in class? 536 00:27:04,993 --> 00:27:06,160 MARGARET TANZOCH: I'm sorry. 537 00:27:06,160 --> 00:27:08,090 Could you say that again? 538 00:27:08,090 --> 00:27:12,953 AUDIENCE: Do you practice to share the project the best projects in class? 539 00:27:12,953 --> 00:27:14,370 MARGARET TANZOCH: Sometimes, I do. 540 00:27:14,370 --> 00:27:15,780 It depends on the class. 541 00:27:15,780 --> 00:27:18,960 I do that a little more in my AP class, where students 542 00:27:18,960 --> 00:27:21,010 are a little more at that point. 543 00:27:21,010 --> 00:27:24,630 So my AP students have generally already had computer science for a year 544 00:27:24,630 --> 00:27:25,660 with me. 545 00:27:25,660 --> 00:27:28,010 So they're a little more aware of what design means. 546 00:27:28,010 --> 00:27:30,010 And they're a little more aware of good choices. 547 00:27:30,010 --> 00:27:32,220 So I will do that sometimes with my AP students. 548 00:27:32,220 --> 00:27:36,900 We'll take a look at a really optimal solution to a problem and kind of walk 549 00:27:36,900 --> 00:27:38,470 through it together. 550 00:27:38,470 --> 00:27:39,480 I do like to do that. 551 00:27:39,480 --> 00:27:41,310 But for some of my introductory students, 552 00:27:41,310 --> 00:27:44,760 again, I'm, at this point, if they could get the program to work, 553 00:27:44,760 --> 00:27:46,290 that's a real success. 554 00:27:46,290 --> 00:27:50,090 And so I'm not always looking at the optimal solution yet. 555 00:27:50,090 --> 00:27:52,650 For AP students, we sometimes do do that. 556 00:27:52,650 --> 00:27:55,770 AUDIENCE: Do practice to share summaries after a number of lecture 557 00:27:55,770 --> 00:27:59,220 or seminars or labs in class? 558 00:27:59,220 --> 00:28:01,980 MARGARET TANZOCH: Again, sometimes, I do and sometimes, I don't. 559 00:28:01,980 --> 00:28:04,590 For my introductory students, because so many students 560 00:28:04,590 --> 00:28:06,490 are working at their own pace. 561 00:28:06,490 --> 00:28:08,800 I'll usually work with small groups of students. 562 00:28:08,800 --> 00:28:11,040 So I may not do a whole class. 563 00:28:11,040 --> 00:28:13,500 If I have a class of 34 students, sometimes, 564 00:28:13,500 --> 00:28:16,680 it's hard to keep everybody's attention, when everybody's not exactly 565 00:28:16,680 --> 00:28:17,852 at the same level. 566 00:28:17,852 --> 00:28:19,560 So what I'll usually do is, in that case, 567 00:28:19,560 --> 00:28:22,810 I'll work with small groups of students who are all working at the same level. 568 00:28:22,810 --> 00:28:24,420 And we'll go through it together. 569 00:28:24,420 --> 00:28:27,852 And so I find that that's a little easier to keep their attention, 570 00:28:27,852 --> 00:28:29,810 if I have four or five students around a table, 571 00:28:29,810 --> 00:28:31,950 and we go through it together, than if I'm 572 00:28:31,950 --> 00:28:35,610 lecturing at the front of the classroom and have 30 students kind 573 00:28:35,610 --> 00:28:38,850 of trying to pay attention to an overview, reviewing the coding 574 00:28:38,850 --> 00:28:40,050 a particular problem. 575 00:28:40,050 --> 00:28:42,095 So I will do that from time to time. 576 00:28:42,095 --> 00:28:44,220 And I usually do that with small groups of students 577 00:28:44,220 --> 00:28:46,050 after they've completed the event. 578 00:28:46,050 --> 00:28:49,820 AUDIENCE: And what about late submission? 579 00:28:49,820 --> 00:28:53,180 MARGARET TANZOCH: So again, because I have such a wide range of students. 580 00:28:53,180 --> 00:28:56,240 And I am-- as I talk about academic honesty. 581 00:28:56,240 --> 00:28:58,370 I'll talk a little about that, as well. 582 00:28:58,370 --> 00:29:00,590 I generally don't take off. 583 00:29:00,590 --> 00:29:02,450 I'm pretty flexible in terms of submissions 584 00:29:02,450 --> 00:29:06,500 because I know that students sometimes have five other AP 585 00:29:06,500 --> 00:29:07,700 classes they may be taking. 586 00:29:07,700 --> 00:29:10,625 They may be working a part time job to help support their family. 587 00:29:10,625 --> 00:29:13,250 They may have other family responsibilities, such as picking up 588 00:29:13,250 --> 00:29:15,500 siblings from school or babysitting. 589 00:29:15,500 --> 00:29:20,270 So I will let them usually submit with no lateness 590 00:29:20,270 --> 00:29:21,963 until the end of the marking period. 591 00:29:21,963 --> 00:29:24,380 Or not the end of the marking period, the end of the unit. 592 00:29:24,380 --> 00:29:26,778 And then after that, unless there's a special excuse, 593 00:29:26,778 --> 00:29:28,820 I'll usually take off a certain amount of credit. 594 00:29:28,820 --> 00:29:30,278 I'll still allow them to submit it. 595 00:29:30,278 --> 00:29:31,460 And they'll get some credit. 596 00:29:31,460 --> 00:29:35,060 But I will take off some credit, possibly 597 00:29:35,060 --> 00:29:37,228 five points for the first few days. 598 00:29:37,228 --> 00:29:38,520 It's like 10 points after that. 599 00:29:38,520 --> 00:29:39,990 It kind of depends year to year. 600 00:29:39,990 --> 00:29:42,490 Year to year I tend to have a different profile of students. 601 00:29:42,490 --> 00:29:44,740 So kind of, again, depending on the needs the students 602 00:29:44,740 --> 00:29:47,390 or the particular population I have, I kind of 603 00:29:47,390 --> 00:29:51,410 customize my policy to really work, to fit the particular students 604 00:29:51,410 --> 00:29:52,582 that I have. 605 00:29:52,582 --> 00:29:55,040 Some students, as well, some teachers, as we'll talk about, 606 00:29:55,040 --> 00:29:57,290 have very strict lateness policies. 607 00:29:57,290 --> 00:30:00,290 And it's certainly easier to keep the whole class going at the same pace 608 00:30:00,290 --> 00:30:01,248 when you're doing that. 609 00:30:01,248 --> 00:30:03,890 And then it does become easier to go over a problem solution 610 00:30:03,890 --> 00:30:08,030 because everybody is already finished with it at that point. 611 00:30:08,030 --> 00:30:15,230 So some teachers will have very specific tardies, like you can have. 612 00:30:15,230 --> 00:30:17,420 And I'll talk about that a little later, as well. 613 00:30:17,420 --> 00:30:20,397 You could have an extra week to submit something maybe once. 614 00:30:20,397 --> 00:30:22,730 And after that, you need to come to me to get permission 615 00:30:22,730 --> 00:30:24,680 to do that, something along those lines. 616 00:30:24,680 --> 00:30:28,100 And they'll have much more specific policies on that. 617 00:30:28,100 --> 00:30:31,460 But then again, given that my students are so all over the place, 618 00:30:31,460 --> 00:30:32,420 in terms of abilities. 619 00:30:32,420 --> 00:30:34,580 And I'm really stressing that I want them 620 00:30:34,580 --> 00:30:36,590 to do their own work on the problem. 621 00:30:36,590 --> 00:30:38,660 I want them to get a real understanding. 622 00:30:38,660 --> 00:30:41,630 Knowing that some students will really struggle with the material, 623 00:30:41,630 --> 00:30:44,210 I prefer to give them extra time to do it on their own 624 00:30:44,210 --> 00:30:48,502 than to feel pressured to copy a solution from somebody else. 625 00:30:48,502 --> 00:30:49,210 AUDIENCE: Oh, no. 626 00:30:49,210 --> 00:30:49,910 Thank you. 627 00:30:49,910 --> 00:30:50,820 Thank you very much. 628 00:30:50,820 --> 00:30:52,220 I really appreciate it. 629 00:30:52,220 --> 00:30:54,200 MARGARET TANZOCH: OK, other questions? 630 00:30:54,200 --> 00:30:58,130 631 00:30:58,130 --> 00:30:59,630 Thiago? 632 00:30:59,630 --> 00:31:01,000 Did you have your hand up? 633 00:31:01,000 --> 00:31:03,640 AUDIENCE: You've talked a little bit about student engagement 634 00:31:03,640 --> 00:31:07,450 and how they will interact with classes. 635 00:31:07,450 --> 00:31:11,830 And [INAUDIBLE] about what types of exercises 636 00:31:11,830 --> 00:31:16,120 should they do like at home and by themselves? 637 00:31:16,120 --> 00:31:16,930 I'm sorry. 638 00:31:16,930 --> 00:31:19,720 And what types of kinds of exercises they 639 00:31:19,720 --> 00:31:24,970 should do at class and with the help of the teacher or professor? 640 00:31:24,970 --> 00:31:27,470 MARGARET TANZOCH: Well, I guess for my high school students, 641 00:31:27,470 --> 00:31:31,730 I generally don't require that they work on their programming problems 642 00:31:31,730 --> 00:31:32,810 for homework. 643 00:31:32,810 --> 00:31:39,050 And again, the reason is that some students don't have high quality Wi-Fi. 644 00:31:39,050 --> 00:31:42,280 They may have limited Wi-Fi minutes that they could use. 645 00:31:42,280 --> 00:31:44,240 And I know that debugging a programming problem 646 00:31:44,240 --> 00:31:47,840 can sometimes-- you can be stuck on it and take hours and hours and hours. 647 00:31:47,840 --> 00:31:51,078 So I don't generally require students to work on their programming problems 648 00:31:51,078 --> 00:31:51,620 for homework. 649 00:31:51,620 --> 00:31:54,160 I give them class time to do that with a lot of support. 650 00:31:54,160 --> 00:31:56,660 The kinds of things I would ask them to do more for homework 651 00:31:56,660 --> 00:31:59,260 might be a writing problem. 652 00:31:59,260 --> 00:32:03,010 So it's something that they usually can do 653 00:32:03,010 --> 00:32:06,360 in a more limited amount of time, maybe do a little research, 654 00:32:06,360 --> 00:32:07,450 or something like that. 655 00:32:07,450 --> 00:32:09,400 So I'll often have them do writing problems. 656 00:32:09,400 --> 00:32:12,700 And for students again, I will post the lecture videos. 657 00:32:12,700 --> 00:32:15,340 And I will suggest if it's possible that they 658 00:32:15,340 --> 00:32:18,190 watch certain chapters of that lecture video for homework, 659 00:32:18,190 --> 00:32:19,690 if they have the ability to do that. 660 00:32:19,690 --> 00:32:23,230 And the ones that do usually do find them extremely beneficial. 661 00:32:23,230 --> 00:32:25,750 Ignatio, did you have another question? 662 00:32:25,750 --> 00:32:28,950 AUDIENCE: Just about collaboration between the students. 663 00:32:28,950 --> 00:32:36,180 Have them some kind of collaboration or [INAUDIBLE] 664 00:32:36,180 --> 00:32:40,680 give them suggestions [INAUDIBLE] the student collaborate between themselves. 665 00:32:40,680 --> 00:32:48,660 And if yes, what kind of software environments 666 00:32:48,660 --> 00:32:52,520 you can use or suggest to use this approach. 667 00:32:52,520 --> 00:33:00,850 Because I was thinking like how researchers work with papers. 668 00:33:00,850 --> 00:33:01,930 I write my paper. 669 00:33:01,930 --> 00:33:12,880 I submit to our colleges that evaluate my work and give me back [INAUDIBLE].. 670 00:33:12,880 --> 00:33:20,947 I was thinking if that is some best way to [INAUDIBLE] in your class. 671 00:33:20,947 --> 00:33:22,780 MARGARET TANZOCH: What software do I are use 672 00:33:22,780 --> 00:33:26,110 for submitting assignments or returning assignments? 673 00:33:26,110 --> 00:33:31,360 AUDIENCE: [INAUDIBLE] kind of platform that you can use when a student submits 674 00:33:31,360 --> 00:33:39,880 [INAUDIBLE] and one, [INAUDIBLE] other student [INAUDIBLE] 675 00:33:39,880 --> 00:33:45,070 and suggest change and give back to the first student. 676 00:33:45,070 --> 00:33:49,070 And [INAUDIBLE] has a chance to improve their own answer. 677 00:33:49,070 --> 00:33:51,267 For. 678 00:33:51,267 --> 00:33:53,350 MARGARET TANZOCH: So for the programming problems, 679 00:33:53,350 --> 00:33:56,470 I do use the Submit50 for pretty much everything. 680 00:33:56,470 --> 00:33:59,463 And what I'll do is I will meet individually. 681 00:33:59,463 --> 00:34:01,630 I'll walk around and I'll look at their submissions. 682 00:34:01,630 --> 00:34:04,990 And I'll usually do one on one conversations with students 683 00:34:04,990 --> 00:34:07,150 about their particular codes. 684 00:34:07,150 --> 00:34:10,030 For some of my AP students, what I might do is then, 685 00:34:10,030 --> 00:34:13,690 I might print out what they've said that it's me electronically 686 00:34:13,690 --> 00:34:17,139 and then make comments on it, either handwritten 687 00:34:17,139 --> 00:34:19,806 or sometimes I might, if it's stories or Google Doc, 688 00:34:19,806 --> 00:34:21,639 in terms of some of the writing assignments, 689 00:34:21,639 --> 00:34:24,014 I might make comments and then also try to meet with them 690 00:34:24,014 --> 00:34:26,710 one on one to go over what those comments are 691 00:34:26,710 --> 00:34:28,699 and how they could improve their work. 692 00:34:28,699 --> 00:34:32,860 So right now, I do find that, for a lot of my students, 693 00:34:32,860 --> 00:34:34,840 having a one on one conversations tends to be 694 00:34:34,840 --> 00:34:39,340 a little bit more effective than using the comments and get help, though. 695 00:34:39,340 --> 00:34:41,620 We'll see next year if we start out remotely, 696 00:34:41,620 --> 00:34:43,760 I'll be looking for other electronic solutions. 697 00:34:43,760 --> 00:34:48,820 So Google classroom is something that we used when we went remote 698 00:34:48,820 --> 00:34:50,389 toward the end of the year. 699 00:34:50,389 --> 00:34:53,250 And so I did give students a lot of feedback on Google classroom, 700 00:34:53,250 --> 00:34:55,730 when they submitted a program or a project, as well. 701 00:34:55,730 --> 00:35:00,640 So they submit their programming projects the CS50, right? 702 00:35:00,640 --> 00:35:02,950 But then any writing assignments or sometimes 703 00:35:02,950 --> 00:35:05,920 even feedback for other programs, I would give them 704 00:35:05,920 --> 00:35:08,140 that feedback back on Google classroom. 705 00:35:08,140 --> 00:35:11,140 So really, quite a number of ways. 706 00:35:11,140 --> 00:35:13,585 But the ones that usually I find work best 707 00:35:13,585 --> 00:35:16,045 are when I would do like one on one meetings with students. 708 00:35:16,045 --> 00:35:18,840 709 00:35:18,840 --> 00:35:20,850 [INAUDIBLE] 710 00:35:20,850 --> 00:35:25,470 AUDIENCE: Do you use any additional resources to supplement learning? 711 00:35:25,470 --> 00:35:28,690 Or do you find that everything from the cs50 712 00:35:28,690 --> 00:35:32,270 is sufficient, even for students struggling? 713 00:35:32,270 --> 00:35:35,700 Are there an external sort of resources, books, 714 00:35:35,700 --> 00:35:37,788 that you might use to help learning? 715 00:35:37,788 --> 00:35:40,080 MARGARET TANZOCH: Well, what I'm going to show tomorrow 716 00:35:40,080 --> 00:35:45,210 is that I create a lot of labs for my students using the cs50 lab platform. 717 00:35:45,210 --> 00:35:50,340 And in the labs, which I'm going to go through tomorrow in my discussion 718 00:35:50,340 --> 00:35:55,560 on the assessments in labs, I have a lot of detailed instructions 719 00:35:55,560 --> 00:35:59,080 on how to do very specific constructs in programming. 720 00:35:59,080 --> 00:36:03,540 So for instance, I'll have, in lab, some students will be working on a program 721 00:36:03,540 --> 00:36:05,730 and they forget how to use command line arguments. 722 00:36:05,730 --> 00:36:07,320 They've seen the mini lesson. 723 00:36:07,320 --> 00:36:09,270 They've seen me do a little demo of it. 724 00:36:09,270 --> 00:36:10,590 But once they're working on their program, 725 00:36:10,590 --> 00:36:13,030 they forget or they don't totally understand how it works. 726 00:36:13,030 --> 00:36:15,480 So I'll have them do a lab. 727 00:36:15,480 --> 00:36:18,510 And the lab will have detailed instructions with a little practice 728 00:36:18,510 --> 00:36:19,650 problem on the right. 729 00:36:19,650 --> 00:36:23,730 And so for students that I'm trying to encourage to be self-sufficient, 730 00:36:23,730 --> 00:36:26,140 or because I may have a lot of students in the classroom 731 00:36:26,140 --> 00:36:28,950 and I can't help them with that very specific thing at that very 732 00:36:28,950 --> 00:36:33,420 specific time, I do find the labs can be really helpful for students 733 00:36:33,420 --> 00:36:34,890 to have the extra support. 734 00:36:34,890 --> 00:36:37,210 I don't require them to buy books. 735 00:36:37,210 --> 00:36:40,320 I do allow them to do research on the internet, 736 00:36:40,320 --> 00:36:43,562 as long as they're not looking up a solution. 737 00:36:43,562 --> 00:36:45,270 And sometimes, there's a little confusion 738 00:36:45,270 --> 00:36:47,812 on what that means, which I'll be talking about a little bit. 739 00:36:47,812 --> 00:36:53,443 But I do encourage them to use resources, the cs50 manual, 740 00:36:53,443 --> 00:36:55,860 depending on what programming language they're working on. 741 00:36:55,860 --> 00:37:02,640 If they're working on the second semester HTML or CSS, 742 00:37:02,640 --> 00:37:04,410 I'll have them use W3 schools. 743 00:37:04,410 --> 00:37:06,370 I'll introduce them to those resources. 744 00:37:06,370 --> 00:37:08,400 So the resources are pretty much online. 745 00:37:08,400 --> 00:37:12,390 And I'll try to find videos, sometimes, when I can. 746 00:37:12,390 --> 00:37:15,647 For my AP class, sometimes there are nice videos 747 00:37:15,647 --> 00:37:18,730 from code.org that talk about how the internet works and things like that. 748 00:37:18,730 --> 00:37:20,570 So sometimes, I'll introduce those. 749 00:37:20,570 --> 00:37:24,180 And even in my lesson, sometimes I'll try to use a video every now and then 750 00:37:24,180 --> 00:37:27,480 to be a little bit engaging, instead of being up there 751 00:37:27,480 --> 00:37:31,260 and talking too much in front of the class, which I find I can only 752 00:37:31,260 --> 00:37:35,080 do for a very limited amount of time, if I want to keep everybody's attention. 753 00:37:35,080 --> 00:37:40,830 So what I'm going to talk about next is handling academic honesty. 754 00:37:40,830 --> 00:37:45,950 So again, many different teachers do this in many different ways. 755 00:37:45,950 --> 00:37:48,410 What we do is at the beginning of the year, 756 00:37:48,410 --> 00:37:51,620 we'll read through the academic honesty policy 757 00:37:51,620 --> 00:37:55,460 that cs50 provides in their curriculum and in their program specs. 758 00:37:55,460 --> 00:37:58,370 And we actually have a class contract that students and their parents 759 00:37:58,370 --> 00:38:02,510 have to sign that also discuss the academic honesty policy. 760 00:38:02,510 --> 00:38:04,680 Several times during the course of the year, 761 00:38:04,680 --> 00:38:09,140 we'll talk about how students can collaborate and help each other while 762 00:38:09,140 --> 00:38:11,420 maintaining academic integrity. 763 00:38:11,420 --> 00:38:14,450 And some of the ways that might look, again, kind of consistent 764 00:38:14,450 --> 00:38:18,110 what you might have seen in the written part of the academic policy 765 00:38:18,110 --> 00:38:22,700 is that, if a student is a class expert, and they're helping somebody else, 766 00:38:22,700 --> 00:38:24,700 and they've already completed the program, 767 00:38:24,700 --> 00:38:28,370 that it wouldn't be academically honest for a student that's 768 00:38:28,370 --> 00:38:30,320 completed their code to show their finished 769 00:38:30,320 --> 00:38:32,240 code to a student who is struggling. 770 00:38:32,240 --> 00:38:34,260 So that's forbidden. 771 00:38:34,260 --> 00:38:36,542 I tell students, if they're helping another student, 772 00:38:36,542 --> 00:38:39,500 they're not allowed to put their hands on that other students keyboard. 773 00:38:39,500 --> 00:38:42,000 Because I've seen sometimes students tempted to kind of, oh, 774 00:38:42,000 --> 00:38:43,220 let me just type this in. 775 00:38:43,220 --> 00:38:46,077 But the student who's struggling has no idea what that student did. 776 00:38:46,077 --> 00:38:48,660 So they're not allowed to type on the other students keyboard. 777 00:38:48,660 --> 00:38:53,720 Everybody has to have hands on and work on their own keyboards. 778 00:38:53,720 --> 00:38:57,230 In terms of collaboration, again, students 779 00:38:57,230 --> 00:38:59,780 that collaborate, I make sure they're at the same level. 780 00:38:59,780 --> 00:39:03,652 I don't want to have students collaborating one that's so much more 781 00:39:03,652 --> 00:39:05,360 advanced or working with a student that's 782 00:39:05,360 --> 00:39:08,030 really struggling because then that collaboration often 783 00:39:08,030 --> 00:39:09,458 ends up going one way. 784 00:39:09,458 --> 00:39:11,750 So I try to have students, when they are collaborating, 785 00:39:11,750 --> 00:39:15,320 pairing up with another student that's pretty much at their same level 786 00:39:15,320 --> 00:39:18,310 and encourage them to kind of work out the solution together, 787 00:39:18,310 --> 00:39:20,930 think up the steps, working in pseudocode, 788 00:39:20,930 --> 00:39:23,420 maybe putting pseudocode the comments first, 789 00:39:23,420 --> 00:39:27,170 and then kind of working back and forth with each other, each in their own IDE, 790 00:39:27,170 --> 00:39:30,260 in most cases, to solve that particular problem. 791 00:39:30,260 --> 00:39:34,520 During the course of the year, I will do one or two pair programming projects. 792 00:39:34,520 --> 00:39:37,490 I don't usually do it until the second semester. 793 00:39:37,490 --> 00:39:39,380 And the reason for that is that, my concern 794 00:39:39,380 --> 00:39:41,172 is that, if students are paired programming 795 00:39:41,172 --> 00:39:44,570 when they're first learning how to manage using the coding 796 00:39:44,570 --> 00:39:47,000 environment, the text editor, the terminal 797 00:39:47,000 --> 00:39:49,130 that it's very possible that one student may end up 798 00:39:49,130 --> 00:39:50,910 not getting all that much out of it. 799 00:39:50,910 --> 00:39:52,730 So at the beginning of the year, I do want 800 00:39:52,730 --> 00:39:55,430 students to all be working in their own environments, 801 00:39:55,430 --> 00:39:56,950 submitting their own work. 802 00:39:56,950 --> 00:39:59,440 And again, with support when they need that. 803 00:39:59,440 --> 00:40:03,770 And really kind of stress alongside with that 804 00:40:03,770 --> 00:40:09,620 that it's really important that students struggle through the problems, 805 00:40:09,620 --> 00:40:10,400 if they need to. 806 00:40:10,400 --> 00:40:12,660 That struggle is how we learn. 807 00:40:12,660 --> 00:40:14,630 We need to have persistence. 808 00:40:14,630 --> 00:40:15,990 That failure is common. 809 00:40:15,990 --> 00:40:18,740 That there is so much failure when we're programming, 810 00:40:18,740 --> 00:40:20,930 we have a special word for it called debugging. 811 00:40:20,930 --> 00:40:22,970 That it's the nature of the way coding works. 812 00:40:22,970 --> 00:40:26,150 And even when I do all my demonstrations, 813 00:40:26,150 --> 00:40:29,450 I will usually make errors, sometimes on purpose, and sometimes 814 00:40:29,450 --> 00:40:30,650 just because I make errors. 815 00:40:30,650 --> 00:40:32,150 I forget my semicolon. 816 00:40:32,150 --> 00:40:37,160 And kind of see how students kind of understand the process programming is 817 00:40:37,160 --> 00:40:39,680 one that's often trial and error. 818 00:40:39,680 --> 00:40:43,340 We'll talk about the difference between using the internet for references 819 00:40:43,340 --> 00:40:46,040 and using the internet for outright solutions. 820 00:40:46,040 --> 00:40:48,620 And again, the reflection that students have 821 00:40:48,620 --> 00:40:51,500 to submit at the end of their program does give me 822 00:40:51,500 --> 00:40:58,610 a good window into how much of the code they actually developed on their own. 823 00:40:58,610 --> 00:41:02,768 Every now and then I'll get a reflection that looks like I finished a program. 824 00:41:02,768 --> 00:41:03,560 It was really easy. 825 00:41:03,560 --> 00:41:07,340 I had a problem, a friend helped me fix it, and now it's done. 826 00:41:07,340 --> 00:41:09,710 So that's usually a clue as to that that student really 827 00:41:09,710 --> 00:41:11,640 had no understanding of what went on with that problem. 828 00:41:11,640 --> 00:41:14,570 And then I'll go have a one to one conversation with them about that. 829 00:41:14,570 --> 00:41:17,110 830 00:41:17,110 --> 00:41:22,200 So in order to try to avoid academic honesty violations, 831 00:41:22,200 --> 00:41:25,230 as I just mentioned, I really do try to stress that, 832 00:41:25,230 --> 00:41:28,470 if students don't put it the work to solve their own problems, 833 00:41:28,470 --> 00:41:32,250 if they're depending too much on help or from their collaborative partner, 834 00:41:32,250 --> 00:41:34,710 that they're really going to get nothing out of the class. 835 00:41:34,710 --> 00:41:36,870 And that, as we go into semester two, if they 836 00:41:36,870 --> 00:41:39,120 don't understand the basics and semester one, 837 00:41:39,120 --> 00:41:41,550 by the time we get to final projects, there's 838 00:41:41,550 --> 00:41:44,010 a pretty good chance they're going to be totally lost. 839 00:41:44,010 --> 00:41:47,070 So I stress it over and over, which is again one of the reasons 840 00:41:47,070 --> 00:41:49,240 why I will give students extra time. 841 00:41:49,240 --> 00:41:52,260 I'll let them know that I much prefer that they have the extra time 842 00:41:52,260 --> 00:41:56,040 to work through the problem on their own or with my support, where 843 00:41:56,040 --> 00:41:58,740 I'll be explaining as we maybe type in code together. 844 00:41:58,740 --> 00:42:02,250 For students that really struggle, we might type in some code 845 00:42:02,250 --> 00:42:05,190 together as I explain it and we write in comments. 846 00:42:05,190 --> 00:42:08,020 And then I have them explain to me what that code does. 847 00:42:08,020 --> 00:42:11,720 So my goal here is that students are learning from something from the class. 848 00:42:11,720 --> 00:42:14,970 Whether they learn it totally on their own because they could just pick it up, 849 00:42:14,970 --> 00:42:16,830 watch the shorts, watch the video, they got it. 850 00:42:16,830 --> 00:42:17,330 They got it. 851 00:42:17,330 --> 00:42:18,090 They're good. 852 00:42:18,090 --> 00:42:22,200 Or if it's through kind of an explanation for me as we go through it. 853 00:42:22,200 --> 00:42:24,750 My goal is that everybody comes out of the class having 854 00:42:24,750 --> 00:42:28,240 some understanding of what it looks like to program in the real world. 855 00:42:28,240 --> 00:42:30,510 And one of the things I love about cs50 is 856 00:42:30,510 --> 00:42:33,990 that we have an environment to code in with a real looking terminal, 857 00:42:33,990 --> 00:42:36,450 so that it is a real world look at what it looks 858 00:42:36,450 --> 00:42:38,230 like to actually create a program. 859 00:42:38,230 --> 00:42:40,440 And I think that students really appreciate that. 860 00:42:40,440 --> 00:42:43,842 861 00:42:43,842 --> 00:42:46,300 But even though I go through all these things occasionally, 862 00:42:46,300 --> 00:42:50,740 there is a student that just doesn't really have the confidence 863 00:42:50,740 --> 00:42:52,570 to start a program on their own. 864 00:42:52,570 --> 00:42:56,678 My classroom is set up where we have about 30 computers 865 00:42:56,678 --> 00:42:57,970 around the outside of the room. 866 00:42:57,970 --> 00:43:01,270 The computers are literally two inches from each other. 867 00:43:01,270 --> 00:43:04,510 And students will sometimes just feel very intimidated 868 00:43:04,510 --> 00:43:07,480 about understanding where to even start typing in 869 00:43:07,480 --> 00:43:10,330 and might have their neighbor help them a little bit, 870 00:43:10,330 --> 00:43:12,820 have a student expert help them a little bit, 871 00:43:12,820 --> 00:43:16,713 to the point that they are not really doing the code on their own. 872 00:43:16,713 --> 00:43:19,630 And again, that's one of those things that the reflection will usually 873 00:43:19,630 --> 00:43:21,310 be reveal to me. 874 00:43:21,310 --> 00:43:23,500 I've occasionally had a student who thought 875 00:43:23,500 --> 00:43:27,550 they did a good thing by finding a live coding tutorial on YouTube because it 876 00:43:27,550 --> 00:43:29,690 taught them how to solve a problem. 877 00:43:29,690 --> 00:43:34,330 And I'll usually have a discussion with them after that. 878 00:43:34,330 --> 00:43:37,628 Because they are feeling, maybe, they didn't just find a get help repository. 879 00:43:37,628 --> 00:43:38,920 They didn't just find the code. 880 00:43:38,920 --> 00:43:40,990 Plus they, actually typed it in themselves. 881 00:43:40,990 --> 00:43:45,320 But we try to have these one on one conversations about that, 882 00:43:45,320 --> 00:43:47,560 when that happens. 883 00:43:47,560 --> 00:43:51,670 Ways that I sometimes discover academic honesty violations 884 00:43:51,670 --> 00:43:55,390 is that, in some cases, I might find a submission that solves 885 00:43:55,390 --> 00:43:57,460 an old version of a cs50 problem. 886 00:43:57,460 --> 00:44:00,970 So I'm lucky in the fact that I've taught this curriculum for five years. 887 00:44:00,970 --> 00:44:02,890 So sometimes, I'll notice that a student tries 888 00:44:02,890 --> 00:44:07,720 to solve Mario by looking at the height of the pyramid is between 1 and 23, 889 00:44:07,720 --> 00:44:10,630 which was the way the program was constructed a few years back. 890 00:44:10,630 --> 00:44:11,890 Now it's between 1 and 8. 891 00:44:11,890 --> 00:44:14,680 So that automatically raises a red flag. 892 00:44:14,680 --> 00:44:17,980 Or perhaps they start a pyramid with two hash tags on top instead of one, 893 00:44:17,980 --> 00:44:20,410 which is also an older version of the problem. 894 00:44:20,410 --> 00:44:24,960 Sometimes, I'll see students do their props with a Print 895 00:44:24,960 --> 00:44:29,560 f before they use a user input function, like getInt or getString, which again, 896 00:44:29,560 --> 00:44:33,760 is an older way that cs50 problems were solved 897 00:44:33,760 --> 00:44:37,282 because using a prompt inside the parentheses as an argument in one 898 00:44:37,282 --> 00:44:40,240 of the user input functions has only been around for about the last two 899 00:44:40,240 --> 00:44:40,870 years. 900 00:44:40,870 --> 00:44:43,180 So there are things, sometimes, that I'll 901 00:44:43,180 --> 00:44:47,020 see that I'll recognize are old ways of doing it, in which case 902 00:44:47,020 --> 00:44:48,820 we'll have that discussion. 903 00:44:48,820 --> 00:44:50,930 I also walk around the classroom. 904 00:44:50,930 --> 00:44:53,950 One of the reasons I like students programming in the classroom 905 00:44:53,950 --> 00:44:56,710 is that, honestly, at home, I have no idea if they 906 00:44:56,710 --> 00:44:58,120 have a parent who's a programmer. 907 00:44:58,120 --> 00:44:58,840 Many of them do. 908 00:44:58,840 --> 00:45:00,640 Not many, some of them do. 909 00:45:00,640 --> 00:45:03,520 Or if they have a friend that's willing to collaborate a little too 910 00:45:03,520 --> 00:45:04,450 much with them. 911 00:45:04,450 --> 00:45:06,910 I like to walk around as they're doing the coding 912 00:45:06,910 --> 00:45:10,000 and just randomly ask students to explain to me what they're working on, 913 00:45:10,000 --> 00:45:13,810 what they're doing, line by line, what their code is doing. 914 00:45:13,810 --> 00:45:16,930 Sometimes I'll notice in-code strange indentations 915 00:45:16,930 --> 00:45:20,200 that might indicate some kind of a copy and paste job. 916 00:45:20,200 --> 00:45:22,930 And that's also just a little bit of an alert. 917 00:45:22,930 --> 00:45:26,560 Again, as I've mentioned a few times, noticing that the reflection provides 918 00:45:26,560 --> 00:45:30,850 absolutely no detail about how their process work, what an aha moment was, 919 00:45:30,850 --> 00:45:35,620 or what kind of struggles they had, or how the algorithm operated, 920 00:45:35,620 --> 00:45:39,430 usually provides insight to me that they probably weren't very academically 921 00:45:39,430 --> 00:45:40,960 honest on that problem. 922 00:45:40,960 --> 00:45:45,040 Sometimes, I'll paste some suspicious looking code into a Google search. 923 00:45:45,040 --> 00:45:47,380 And I'll immediately see that whole solution 924 00:45:47,380 --> 00:45:50,180 come up somewhere on the internet. 925 00:45:50,180 --> 00:45:52,450 Some teachers do use Compare50. 926 00:45:52,450 --> 00:45:54,250 And I think it's a terrific tool. 927 00:45:54,250 --> 00:45:57,160 I have not used it up until now, for various reasons. 928 00:45:57,160 --> 00:45:59,560 I may try it out next year. 929 00:45:59,560 --> 00:46:02,050 I usually get so caught up kind of designing my class 930 00:46:02,050 --> 00:46:05,743 that I sometimes run out of time to try to use all the tools. 931 00:46:05,743 --> 00:46:07,660 And also, at the beginning of the school year, 932 00:46:07,660 --> 00:46:12,220 and since I do allow some collaboration, a lot of the programming solutions 933 00:46:12,220 --> 00:46:14,030 are going to look very similar. 934 00:46:14,030 --> 00:46:16,570 So again, I depend a little more on the reflections 935 00:46:16,570 --> 00:46:18,640 and on the personal relationships with students. 936 00:46:18,640 --> 00:46:21,070 But I think Compare50 is a great tool. 937 00:46:21,070 --> 00:46:24,700 And I know some teachers use it very, very effectively. 938 00:46:24,700 --> 00:46:30,130 I also know that some teachers will use the file revision history in the IDE 939 00:46:30,130 --> 00:46:33,550 to see if an entire program is pasted in like one step. 940 00:46:33,550 --> 00:46:36,970 And that's a great thing that you can see in the final revision history. 941 00:46:36,970 --> 00:46:39,640 Some cs50 teachers will have all of their students 942 00:46:39,640 --> 00:46:41,360 share their workspaces with them. 943 00:46:41,360 --> 00:46:43,360 And then they'll go through each of them and see 944 00:46:43,360 --> 00:46:45,490 how they indeed develop their program. 945 00:46:45,490 --> 00:46:47,410 Again, if I had a small class of students, 946 00:46:47,410 --> 00:46:49,035 I think that would be a great solution. 947 00:46:49,035 --> 00:46:53,970 But I just can't have 150 students share their workspace with me. 948 00:46:53,970 --> 00:46:56,560 And they get a sense of how to move forward with that. 949 00:46:56,560 --> 00:46:59,800 So what I will use file revision history for is, sometimes 950 00:46:59,800 --> 00:47:02,890 if I'm suspecting that a student copied and pasted code, 951 00:47:02,890 --> 00:47:04,510 I'll go over them in class. 952 00:47:04,510 --> 00:47:07,500 And we'll take a look at their file revision history together. 953 00:47:07,500 --> 00:47:09,067 And that might be very revealing. 954 00:47:09,067 --> 00:47:11,650 Usually, they'll try to have an excuse like, well, I developed 955 00:47:11,650 --> 00:47:13,900 my program in a Google doc first. 956 00:47:13,900 --> 00:47:16,550 And then I pasted it in the IDE. 957 00:47:16,550 --> 00:47:20,630 It's like, hmm, that doesn't sound like a very good way to go. 958 00:47:20,630 --> 00:47:24,320 Right, so what do I do when I find academic honesty violations? 959 00:47:24,320 --> 00:47:27,710 So I'll have a one on one conversation with the student. 960 00:47:27,710 --> 00:47:31,050 And honestly, in most cases, they'll usually admit it. 961 00:47:31,050 --> 00:47:33,692 If it's an introductory student, for their first offense, 962 00:47:33,692 --> 00:47:36,650 I don't necessarily tell them this at the beginning of the school year. 963 00:47:36,650 --> 00:47:40,760 But for the first offense, I'll often have them delete their entire program 964 00:47:40,760 --> 00:47:44,107 and write the whole program again on their own, or maybe with my help. 965 00:47:44,107 --> 00:47:44,940 And I'll watch them. 966 00:47:44,940 --> 00:47:47,600 I'll kind of keep a special eye on them as they're working on it. 967 00:47:47,600 --> 00:47:49,433 And in almost every case, when that happens, 968 00:47:49,433 --> 00:47:52,270 the code that they develop themselves is usually 969 00:47:52,270 --> 00:47:55,820 of code that looks like maybe simpler code, which 970 00:47:55,820 --> 00:47:59,630 I would expect that student to be developing the beginning of the year. 971 00:47:59,630 --> 00:48:04,160 Sometimes, I'll also just identify that a student uses very sophisticated code, 972 00:48:04,160 --> 00:48:06,410 or some kind of programming trick that we haven't even 973 00:48:06,410 --> 00:48:08,830 come close to talking about, like pointers 974 00:48:08,830 --> 00:48:11,840 for strings or things like that instead of a string data type, 975 00:48:11,840 --> 00:48:16,730 which usually indicates to me that it's probably not their code. 976 00:48:16,730 --> 00:48:20,040 If it's an AP student who I've already had in my class for a year, 977 00:48:20,040 --> 00:48:22,970 or if it's a second offense, I'll give them a 0 on that problem. 978 00:48:22,970 --> 00:48:25,062 And that if it ever happens again, I would 979 00:48:25,062 --> 00:48:27,020 call home and have a meeting with their student 980 00:48:27,020 --> 00:48:28,840 and their parents and an administrator. 981 00:48:28,840 --> 00:48:31,340 But honestly, after the student gets a 0 for the first time, 982 00:48:31,340 --> 00:48:34,070 it pretty much never really happens again. 983 00:48:34,070 --> 00:48:36,410 And I've also found that, in most cases, in having 984 00:48:36,410 --> 00:48:38,480 the discussion with a student, there's usually 985 00:48:38,480 --> 00:48:41,630 some underlying challenge that they're facing, 986 00:48:41,630 --> 00:48:45,320 which is why they just feel pressured to get a little too 987 00:48:45,320 --> 00:48:47,210 much help for their program. 988 00:48:47,210 --> 00:48:49,370 Again, which is why I don't have strict deadlines. 989 00:48:49,370 --> 00:48:53,240 I try to keep it open, so that students have the time to work on it 990 00:48:53,240 --> 00:48:56,240 and to actually develop their own code. 991 00:48:56,240 --> 00:48:58,330 AUDIENCE: Margaret? 992 00:48:58,330 --> 00:49:02,070 Sorry, my mic was open so maybe I was encouraged to ask a question. 993 00:49:02,070 --> 00:49:04,220 MARGARET TANZOCH: No, please do. 994 00:49:04,220 --> 00:49:08,910 AUDIENCE: Yeah, and I see, because you have high school students 995 00:49:08,910 --> 00:49:14,220 and you're in a formal setting and these things are critical. 996 00:49:14,220 --> 00:49:18,270 And they are also at the age where you want to teach them work ethic. 997 00:49:18,270 --> 00:49:20,520 For our students, they're a little older. 998 00:49:20,520 --> 00:49:22,870 And they're also self motivated. 999 00:49:22,870 --> 00:49:24,450 They really want to learn. 1000 00:49:24,450 --> 00:49:27,600 And they have a project that they want to build later. 1001 00:49:27,600 --> 00:49:31,390 So it's mostly about the skills for them. 1002 00:49:31,390 --> 00:49:35,742 So at what point it reaches diminishing returns? 1003 00:49:35,742 --> 00:49:38,700 By that, I mean, like if you're going to struggle with a problem trying 1004 00:49:38,700 --> 00:49:45,120 to solve it, is it two hours, where you need to use another skill 1005 00:49:45,120 --> 00:49:51,420 and learn how to phrase your question and ask for a code review on a form? 1006 00:49:51,420 --> 00:49:54,330 Because I think that that's a great skill. 1007 00:49:54,330 --> 00:49:57,330 When everything else fails, you need to know how to ask questions 1008 00:49:57,330 --> 00:50:01,290 and how to build a community, also answer questions. 1009 00:50:01,290 --> 00:50:03,390 That way, people respond to you really well. 1010 00:50:03,390 --> 00:50:04,830 And your writing is really good. 1011 00:50:04,830 --> 00:50:08,790 And you get this goodwill within the community. 1012 00:50:08,790 --> 00:50:09,580 So that's one. 1013 00:50:09,580 --> 00:50:14,850 And then for like Python, we use cheat sheets. 1014 00:50:14,850 --> 00:50:19,500 So is there anything like that is useful for high school students, where 1015 00:50:19,500 --> 00:50:23,170 they have a cheat that gives them the most used things. 1016 00:50:23,170 --> 00:50:25,013 And it would have them. 1017 00:50:25,013 --> 00:50:28,150 And then, at some point, they memorize it and not need it anymore. 1018 00:50:28,150 --> 00:50:31,680 MARGARET TANZOCH: Yeah, I do create like a syntax sheet that I hand out 1019 00:50:31,680 --> 00:50:34,140 or I sometimes just do it electronically now. 1020 00:50:34,140 --> 00:50:37,590 I also have anchor charts all over the classroom with all of those, 1021 00:50:37,590 --> 00:50:38,635 like here is a for loop. 1022 00:50:38,635 --> 00:50:39,510 Here is a while loop. 1023 00:50:39,510 --> 00:50:43,180 Here is how to use arguments and that type of thing. 1024 00:50:43,180 --> 00:50:46,980 So I will have those kind of references available both on the website 1025 00:50:46,980 --> 00:50:52,260 as a syntax reminder sheet, as well as anchor charts all over the classroom. 1026 00:50:52,260 --> 00:50:56,430 Because I don't necessarily expect students to just automatically memorize 1027 00:50:56,430 --> 00:50:57,105 syntax. 1028 00:50:57,105 --> 00:50:59,730 The idea that I'm really pushing for is to memorize ideas-- not 1029 00:50:59,730 --> 00:51:05,410 to memorize, but to understand ideas in programming, computational thinking, 1030 00:51:05,410 --> 00:51:05,910 right? 1031 00:51:05,910 --> 00:51:08,077 To understand how to break down a problem into steps 1032 00:51:08,077 --> 00:51:10,180 and to think about how to solve it. 1033 00:51:10,180 --> 00:51:14,040 So I do provide them plenty of assistance on syntax. 1034 00:51:14,040 --> 00:51:16,380 And even when I give quizzes, I will allow 1035 00:51:16,380 --> 00:51:19,350 them to use a syntax sheet and a page of notes 1036 00:51:19,350 --> 00:51:22,380 that they've created on their own, which they can then 1037 00:51:22,380 --> 00:51:24,180 use to help them with that quiz. 1038 00:51:24,180 --> 00:51:28,963 And that page of notes actually is like a study guide, one page, two sides, 1039 00:51:28,963 --> 00:51:29,880 has to be handwritten. 1040 00:51:29,880 --> 00:51:31,600 It can't be printed out from online. 1041 00:51:31,600 --> 00:51:32,730 They have to write it. 1042 00:51:32,730 --> 00:51:36,100 The process of creating that is actually a great learning experience for them. 1043 00:51:36,100 --> 00:51:38,560 So I will allow them to do that, as well. 1044 00:51:38,560 --> 00:51:43,210 So yeah, I mean it's again, I have students 1045 00:51:43,210 --> 00:51:45,370 of such a wide range of abilities. 1046 00:51:45,370 --> 00:51:48,590 And I want to make it possible for everybody to, 1047 00:51:48,590 --> 00:51:52,850 at the very least pass my class, if not do well, if they put in the effort. 1048 00:51:52,850 --> 00:51:56,650 So what I'll do with students that really struggle is 1049 00:51:56,650 --> 00:52:00,147 that I will hopefully have enough resources out there that most 1050 00:52:00,147 --> 00:52:02,230 students can work on their own, or with an expert. 1051 00:52:02,230 --> 00:52:05,680 And I'll pull together maybe four or five students at a table in the middle. 1052 00:52:05,680 --> 00:52:07,903 And we'll work out some code together. 1053 00:52:07,903 --> 00:52:09,820 And we'll literally go through the first step. 1054 00:52:09,820 --> 00:52:14,770 Let's create a do while loop to validate user input. 1055 00:52:14,770 --> 00:52:15,880 How do we do that? 1056 00:52:15,880 --> 00:52:17,890 And I'll ask them, how do we start it? 1057 00:52:17,890 --> 00:52:18,970 How do we write that? 1058 00:52:18,970 --> 00:52:20,560 What's the condition going to be? 1059 00:52:20,560 --> 00:52:22,420 And we'll do that together, interactively. 1060 00:52:22,420 --> 00:52:24,503 And I'll type it up and project that on the board. 1061 00:52:24,503 --> 00:52:26,462 And then they'll type that, usually, at the hub 1062 00:52:26,462 --> 00:52:28,150 of the school laptops they'll grab. 1063 00:52:28,150 --> 00:52:31,990 And they'll sit around the table and type that in with me. 1064 00:52:31,990 --> 00:52:34,870 And so the goal, again, is to really hold 1065 00:52:34,870 --> 00:52:37,120 their hands to support them when they really need it. 1066 00:52:37,120 --> 00:52:42,625 And I also do walkthroughs on problems that go beyond kind of cs50 walkthrough 1067 00:52:42,625 --> 00:52:44,125 that give them a lot more structure. 1068 00:52:44,125 --> 00:52:46,780 And we might work out pseudocode together for a problem, 1069 00:52:46,780 --> 00:52:49,330 so that students that struggle know where to get started. 1070 00:52:49,330 --> 00:52:51,025 So I do try to provide a lot of support. 1071 00:52:51,025 --> 00:52:54,030 1072 00:52:54,030 --> 00:52:55,270 So I see some more questions. 1073 00:52:55,270 --> 00:52:55,770 Patricia? 1074 00:52:55,770 --> 00:52:58,280 1075 00:52:58,280 --> 00:53:02,750 AUDIENCE: Hi, thank you, Margaret, for your time, for your presentation. 1076 00:53:02,750 --> 00:53:05,770 I am just wondering if you can please explain to us a little 1077 00:53:05,770 --> 00:53:08,700 how you organize your semester. 1078 00:53:08,700 --> 00:53:13,240 Because you're explaining about how you grade and how 1079 00:53:13,240 --> 00:53:15,430 you would handle some situations. 1080 00:53:15,430 --> 00:53:21,590 But I am just wondering, when you have so many students, how many hours do you 1081 00:53:21,590 --> 00:53:24,470 have in classes during the semester? 1082 00:53:24,470 --> 00:53:27,620 If you have some labs, if you can tell us a little bit, 1083 00:53:27,620 --> 00:53:32,250 so we can imagine could you apply the things we are discussing now. 1084 00:53:32,250 --> 00:53:32,855 Thanks. 1085 00:53:32,855 --> 00:53:33,980 MARGARET TANZOCH: Sure, no. 1086 00:53:33,980 --> 00:53:34,710 I appreciate that. 1087 00:53:34,710 --> 00:53:35,210 Thank you. 1088 00:53:35,210 --> 00:53:36,690 It's a great question. 1089 00:53:36,690 --> 00:53:41,090 So I meet my students five days a week for 45 minutes a day. 1090 00:53:41,090 --> 00:53:42,920 There's no special section that's a lab. 1091 00:53:42,920 --> 00:53:45,630 It's just a 45 minute class, five times a week. 1092 00:53:45,630 --> 00:53:47,915 And what I do is that I do a very short-- 1093 00:53:47,915 --> 00:53:50,750 I would always have a warm up on the board when they come in. 1094 00:53:50,750 --> 00:53:52,370 And that might be on various different topics. 1095 00:53:52,370 --> 00:53:54,500 So that they come in, they pull out their notebooks. 1096 00:53:54,500 --> 00:53:56,000 And they write with pen and pencils. 1097 00:53:56,000 --> 00:53:57,813 They don't start out with their computers. 1098 00:53:57,813 --> 00:53:59,480 We have tables in the middle of a class. 1099 00:53:59,480 --> 00:54:01,970 And the computers are around the outside of the classroom 1100 00:54:01,970 --> 00:54:03,343 that they move to later on. 1101 00:54:03,343 --> 00:54:04,760 So we'll start out with a problem. 1102 00:54:04,760 --> 00:54:07,910 It might be the beginning of the year to just convert a binary number, 1103 00:54:07,910 --> 00:54:10,940 maybe to refresh a concept from the day before. 1104 00:54:10,940 --> 00:54:13,940 Sometimes, it's a discussion question, like talk 1105 00:54:13,940 --> 00:54:17,510 to your partner about how we can collaborate with academic integrity. 1106 00:54:17,510 --> 00:54:20,620 It might be a little code on the board. 1107 00:54:20,620 --> 00:54:23,620 Determine what this code is doing, line by line. 1108 00:54:23,620 --> 00:54:25,730 I like to give them snippets of code to see 1109 00:54:25,730 --> 00:54:28,183 what code and good well-structured code looks like, 1110 00:54:28,183 --> 00:54:29,350 that they have to interpret. 1111 00:54:29,350 --> 00:54:31,260 So it might be something like that. 1112 00:54:31,260 --> 00:54:33,770 So I'll always start out with like about a 5 minute warm up. 1113 00:54:33,770 --> 00:54:36,290 And then I'll do a very short mini lesson 1114 00:54:36,290 --> 00:54:40,040 and sometimes, a little demonstration of that particular topic. 1115 00:54:40,040 --> 00:54:40,880 I keep it short. 1116 00:54:40,880 --> 00:54:44,750 So my mini lesson probably never goes more than 10 minutes. 1117 00:54:44,750 --> 00:54:48,470 And I might, for instance, talk about a loop, 1118 00:54:48,470 --> 00:54:50,360 like why do we use a while loop, right? 1119 00:54:50,360 --> 00:54:51,440 And how does it work. 1120 00:54:51,440 --> 00:54:54,810 And then I might write a little bit of code and kind of interact 1121 00:54:54,810 --> 00:54:59,120 because I think it is helpful for students to see actual life coding. 1122 00:54:59,120 --> 00:55:02,360 They don't code with me because their computers 1123 00:55:02,360 --> 00:55:04,910 are facing the outside of the room. 1124 00:55:04,910 --> 00:55:09,050 When I was teaching remotely on Zoom, I did have students actually code along 1125 00:55:09,050 --> 00:55:11,600 with me, which I actually really like. 1126 00:55:11,600 --> 00:55:13,400 But I can't really do that in the classroom 1127 00:55:13,400 --> 00:55:15,210 because there's no way I could get their attention when 1128 00:55:15,210 --> 00:55:17,690 they're facing each other and facing the back of the room. 1129 00:55:17,690 --> 00:55:20,280 So I kind of take that into consideration. 1130 00:55:20,280 --> 00:55:21,680 So I'll do a short code along. 1131 00:55:21,680 --> 00:55:24,680 And I honestly don't go on for too long because I'll lose most students. 1132 00:55:24,680 --> 00:55:26,900 Their attention spans are not really that long. 1133 00:55:26,900 --> 00:55:30,380 So I usually try to give them between the lesson 1134 00:55:30,380 --> 00:55:33,200 and the code along, maybe 15 minutes tops. 1135 00:55:33,200 --> 00:55:36,920 And then I'll have them spend about a 1/2 hour, 20 minutes to a 1/2 hour 1136 00:55:36,920 --> 00:55:39,110 on the computers, working on their program. 1137 00:55:39,110 --> 00:55:41,480 And during that time, what they'll do-- and I'll 1138 00:55:41,480 --> 00:55:43,340 introduce them to the program spec. 1139 00:55:43,340 --> 00:55:45,620 I'll show them where some of the resources are. 1140 00:55:45,620 --> 00:55:47,870 And some of them will be sitting around the classroom, 1141 00:55:47,870 --> 00:55:51,440 maybe watching the video walkthrough for that program. 1142 00:55:51,440 --> 00:55:56,600 Or they might be sitting and they may be watching one of Brian's walkthroughs. 1143 00:55:56,600 --> 00:55:59,360 Or they might be watching one of Doug's shorts. 1144 00:55:59,360 --> 00:56:02,300 Or they might be watching a chapter of the lecture video. 1145 00:56:02,300 --> 00:56:04,490 Some of them might be starting with their coding. 1146 00:56:04,490 --> 00:56:07,070 Some of them might be sitting together with another student, 1147 00:56:07,070 --> 00:56:08,750 working at pseudocode in advance. 1148 00:56:08,750 --> 00:56:13,970 So it's a very diversified group and a lot of differentiation 1149 00:56:13,970 --> 00:56:15,930 in terms of what students need. 1150 00:56:15,930 --> 00:56:19,070 And yeah, then at the end of the class, I'll 1151 00:56:19,070 --> 00:56:21,020 sometimes stop five minutes before the end. 1152 00:56:21,020 --> 00:56:25,310 It kind of depends on how much time we have for coding that day. 1153 00:56:25,310 --> 00:56:28,460 I might ask them to have a little share out, what did you accomplish today? 1154 00:56:28,460 --> 00:56:31,250 Sometimes, I'll have them write a little reflection on a Post-It. 1155 00:56:31,250 --> 00:56:33,620 And then that might go up on the poster that I 1156 00:56:33,620 --> 00:56:35,840 have to correspond to that programming problem, 1157 00:56:35,840 --> 00:56:38,000 so that I can also see what questions they have 1158 00:56:38,000 --> 00:56:39,950 that maybe they need some extra help with. 1159 00:56:39,950 --> 00:56:42,350 And then the next day I might ask one of the experts 1160 00:56:42,350 --> 00:56:46,520 for that program to go over and help that particular student. 1161 00:56:46,520 --> 00:56:49,520 And then I think, also, students see that other students have questions, 1162 00:56:49,520 --> 00:56:51,920 too, that it's normal to have questions. 1163 00:56:51,920 --> 00:56:55,200 It's normal to get stuck on stuff, that this is the part of the process. 1164 00:56:55,200 --> 00:56:56,720 This is how it goes for everybody. 1165 00:56:56,720 --> 00:56:59,925 So that they don't feel like, oh, I compiled my code and. 1166 00:56:59,925 --> 00:57:00,550 It didn't work. 1167 00:57:00,550 --> 00:57:02,190 I just can't do this. 1168 00:57:02,190 --> 00:57:05,998 So I try very hard to make it a real culture community, where 1169 00:57:05,998 --> 00:57:07,290 students could help each other. 1170 00:57:07,290 --> 00:57:10,580 But again, constantly stressing that you're helping each other 1171 00:57:10,580 --> 00:57:12,817 with academic integrity. 1172 00:57:12,817 --> 00:57:14,275 I hope that answered your question. 1173 00:57:14,275 --> 00:57:17,520 1174 00:57:17,520 --> 00:57:18,020 All right. 1175 00:57:18,020 --> 00:57:22,600 AUDIENCE: Hi, Professor, I really appreciate your presentation. 1176 00:57:22,600 --> 00:57:23,890 You are so clear. 1177 00:57:23,890 --> 00:57:30,790 But what about any complaint after you have released scores to students? 1178 00:57:30,790 --> 00:57:34,660 How do you proceed, based on your experience, please? 1179 00:57:34,660 --> 00:57:36,910 MARGARET TANZOCH: If they complain about their scores? 1180 00:57:36,910 --> 00:57:41,500 Well, in terms of the cs50, the Submit50, 1181 00:57:41,500 --> 00:57:44,738 they can see very clearly what their score is and why they got it, right? 1182 00:57:44,738 --> 00:57:46,030 They could look at the Check50. 1183 00:57:46,030 --> 00:57:47,322 They could look at the Style50. 1184 00:57:47,322 --> 00:57:50,860 So their program score is based on that. 1185 00:57:50,860 --> 00:57:53,860 The rest of the score depends on the reflection. 1186 00:57:53,860 --> 00:57:56,935 And so if they're not happy about the score with that reflection, 1187 00:57:56,935 --> 00:57:58,810 I will have a one on one discussion with them 1188 00:57:58,810 --> 00:58:02,320 that they have to provide more detail, or they didn't answer a question. 1189 00:58:02,320 --> 00:58:07,620 So for the most part, students are not shocked at their scores. 1190 00:58:07,620 --> 00:58:10,120 Sometimes, students will submit stuff, and they don't really 1191 00:58:10,120 --> 00:58:11,620 check the score they get. 1192 00:58:11,620 --> 00:58:14,920 And then a couple weeks later, it's like, oh, my gosh, I got such a bad. 1193 00:58:14,920 --> 00:58:16,750 Score could I prove my grade? 1194 00:58:16,750 --> 00:58:18,820 So it kind of depends where we are in terms 1195 00:58:18,820 --> 00:58:20,985 of the unit and the marking period. 1196 00:58:20,985 --> 00:58:22,110 Sometimes, I'll allow them. 1197 00:58:22,110 --> 00:58:25,870 I will usually allow them to resubmit it if it's still within the unit. 1198 00:58:25,870 --> 00:58:27,850 And they are allowed to improve their grade. 1199 00:58:27,850 --> 00:58:32,570 And if it's already at the end of the marking period, it is what it is. 1200 00:58:32,570 --> 00:58:36,130 Sometimes, I will allow them to do extra credit problems, like a writing 1201 00:58:36,130 --> 00:58:39,970 assignment, as I mentioned earlier, on a particular topic that's relevant to us 1202 00:58:39,970 --> 00:58:41,480 at that moment. 1203 00:58:41,480 --> 00:58:45,760 And then I will maybe substitute that to their lowest score on their programming 1204 00:58:45,760 --> 00:58:46,810 assignment. 1205 00:58:46,810 --> 00:58:49,600 Because some students are going to struggle with it. 1206 00:58:49,600 --> 00:58:52,662 Everybody is not going to get 100% on every program. 1207 00:58:52,662 --> 00:58:55,120 But again, for students that are willing to put in the work 1208 00:58:55,120 --> 00:58:58,720 and are willing to put in the effort, I will 1209 00:58:58,720 --> 00:59:02,440 try to come up with a way of helping them improve their grade. 1210 00:59:02,440 --> 00:59:04,330 And for the most part, it works out. 1211 00:59:04,330 --> 00:59:07,692 Most students get either an A or B in the class. 1212 00:59:07,692 --> 00:59:10,150 I don't have a whole lot of students that struggle to pass. 1213 00:59:10,150 --> 00:59:13,480 1214 00:59:13,480 --> 00:59:17,040 I hope that answered your question. 1215 00:59:17,040 --> 00:59:20,170 AUDIENCE: Can I ask questions now? 1216 00:59:20,170 --> 00:59:28,970 In the previous line, you said that you use file revision history to check. 1217 00:59:28,970 --> 00:59:32,160 I didn't quite understand how can we do that. 1218 00:59:32,160 --> 00:59:33,650 Well, what kind of tool to use. 1219 00:59:33,650 --> 00:59:35,150 Is it submit.cs50.io? 1220 00:59:35,150 --> 00:59:37,940 1221 00:59:37,940 --> 00:59:39,930 MARGARET TANZOCH: So when you're in the IDE-- 1222 00:59:39,930 --> 00:59:41,680 and I'll be doing a live demo in tomorrow. 1223 00:59:41,680 --> 00:59:44,230 So if you're online tomorrow, I'll be happy to demonstrate 1224 00:59:44,230 --> 00:59:46,460 what that looks like. 1225 00:59:46,460 --> 00:59:49,100 In the IDE, there is, under the file tab, 1226 00:59:49,100 --> 00:59:51,248 there's something called file revision history. 1227 00:59:51,248 --> 00:59:52,790 And there's a slider that's provided. 1228 00:59:52,790 --> 00:59:54,630 And it allows you to go back in time. 1229 00:59:54,630 --> 00:59:58,160 You actually see timestamps for exactly the way you typed in your code 1230 00:59:58,160 --> 01:00:01,170 from the very beginning at the time when you created your file. 1231 01:00:01,170 --> 01:00:06,320 So if I slide back and I see a full program developed and then 1232 01:00:06,320 --> 01:00:10,110 one minute later, nothing in the code, the only way 1233 01:00:10,110 --> 01:00:14,250 that that code could get into that file is by a copy and paste, right? 1234 01:00:14,250 --> 01:00:18,780 You can't type in code and have it all appear in one step. 1235 01:00:18,780 --> 01:00:22,400 So that's one of the things that I might look at to see how the student, 1236 01:00:22,400 --> 01:00:24,868 the process that they use to develop their code. 1237 01:00:24,868 --> 01:00:26,160 So it's a really valuable tool. 1238 01:00:26,160 --> 01:00:29,540 And it's great, too, for when you're programming, if you mess something up, 1239 01:00:29,540 --> 01:00:32,210 you could go back and reverse your code to a previous place, 1240 01:00:32,210 --> 01:00:35,490 so that you haven't lost old versions of it. 1241 01:00:35,490 --> 01:00:38,668 And right now, that works in the IDE environment. 1242 01:00:38,668 --> 01:00:41,460 Not in the labs, but I understand next year that may be integrated. 1243 01:00:41,460 --> 01:00:42,060 So we'll see. 1244 01:00:42,060 --> 01:00:43,310 That's a really terrific tool. 1245 01:00:43,310 --> 01:00:45,355 1246 01:00:45,355 --> 01:00:47,980 And if you remind me tomorrow, I'll be happy to demonstrate it, 1247 01:00:47,980 --> 01:00:51,160 when we do the lives, the kind of demo on how 1248 01:00:51,160 --> 01:00:55,070 submissions work from a student and a teacher perspective. 1249 01:00:55,070 --> 01:00:56,240 I see another hand raised. 1250 01:00:56,240 --> 01:00:58,460 Is that [INAUDIBLE]? 1251 01:00:58,460 --> 01:00:59,690 And again please correct me. 1252 01:00:59,690 --> 01:01:00,660 AUDIENCE: Yes. 1253 01:01:00,660 --> 01:01:01,650 [INAUDIBLE] 1254 01:01:01,650 --> 01:01:02,900 MARGARET TANZOCH: [INAUDIBLE]. 1255 01:01:02,900 --> 01:01:04,600 Nice to meet you. 1256 01:01:04,600 --> 01:01:08,810 AUDIENCE: Hi, I'm enjoying your presentation. 1257 01:01:08,810 --> 01:01:10,560 I just have one question. 1258 01:01:10,560 --> 01:01:15,840 You mentioned the warmup at the beginning of the class, 1259 01:01:15,840 --> 01:01:20,490 and that your class takes about 45 minutes, five days per week. 1260 01:01:20,490 --> 01:01:24,770 So I was wondering that a student who is struggling, 1261 01:01:24,770 --> 01:01:27,130 when that student comes to class like we said, 1262 01:01:27,130 --> 01:01:30,150 on Monday how much time would a struggling student 1263 01:01:30,150 --> 01:01:35,100 would have to spend like in a night to come back the next day's class 1264 01:01:35,100 --> 01:01:38,103 to continue with the followup? 1265 01:01:38,103 --> 01:01:41,270 MARGARET TANZOCH: Well, again, I don't-- how much time would I have to spend 1266 01:01:41,270 --> 01:01:43,370 working on their own for homework, you mean? 1267 01:01:43,370 --> 01:01:47,100 1268 01:01:47,100 --> 01:01:52,570 AUDIENCE: Not really homework, but actually, it might be a great lesson, 1269 01:01:52,570 --> 01:01:55,350 but that student may be struggling. 1270 01:01:55,350 --> 01:02:01,530 And so it will need to go back and revise and recapture what was done. 1271 01:02:01,530 --> 01:02:06,040 So your 45 minutes might be more like 2 hours for that student, 1272 01:02:06,040 --> 01:02:08,100 talking about a struggling student. 1273 01:02:08,100 --> 01:02:11,100 MARGARET TANZOCH: Yes, and I do have those type of students, absolutely. 1274 01:02:11,100 --> 01:02:16,090 So what I'll do is, then, I will do a small group work with those students. 1275 01:02:16,090 --> 01:02:19,740 So I'll pull together a group of students, three or four students, 1276 01:02:19,740 --> 01:02:20,610 possibly. 1277 01:02:20,610 --> 01:02:22,260 We'll sit around the table. 1278 01:02:22,260 --> 01:02:24,717 And my computer will project to the screen, 1279 01:02:24,717 --> 01:02:26,550 while other students are on their computers, 1280 01:02:26,550 --> 01:02:28,860 facing like the sides of the back of the room. 1281 01:02:28,860 --> 01:02:30,610 I'll sit through with those students. 1282 01:02:30,610 --> 01:02:35,160 And I'll work through maybe a little bit more of a live coding example, 1283 01:02:35,160 --> 01:02:39,570 for instance, on how to use operators, how do we assign values to variables, 1284 01:02:39,570 --> 01:02:42,810 kind of break it down very, very step by step. 1285 01:02:42,810 --> 01:02:46,020 I may actually start working on a program with them. 1286 01:02:46,020 --> 01:02:47,902 So I may have them open up one of-- 1287 01:02:47,902 --> 01:02:49,360 we have a couple of school laptops. 1288 01:02:49,360 --> 01:02:50,550 They'll sit around. 1289 01:02:50,550 --> 01:02:54,420 And they may type in the first couple lines of code with me. 1290 01:02:54,420 --> 01:02:57,570 And I'll allow them, in a very small group environment, 1291 01:02:57,570 --> 01:03:00,477 they're much more comfortable asking questions. 1292 01:03:00,477 --> 01:03:02,310 It's true, in a full class environment, they 1293 01:03:02,310 --> 01:03:05,040 may feel intimidated, particularly because sometimes, you'll 1294 01:03:05,040 --> 01:03:07,770 have students that struggle to pass algebra 1295 01:03:07,770 --> 01:03:11,190 sitting right next to a student that's taking calculus. 1296 01:03:11,190 --> 01:03:15,915 And so they may feel very embarrassed and feel like, I just can't do this. 1297 01:03:15,915 --> 01:03:17,290 Everybody has this other than me. 1298 01:03:17,290 --> 01:03:19,675 So that's why I've tried very hard to make sure 1299 01:03:19,675 --> 01:03:22,800 that we have a classroom culture of students being accepting of each other. 1300 01:03:22,800 --> 01:03:26,487 And I'll work, again, with small groups of two or three students, 1301 01:03:26,487 --> 01:03:28,320 so that they feel a little more comfortable. 1302 01:03:28,320 --> 01:03:32,170 And I'll go over that lesson with them over and over again. 1303 01:03:32,170 --> 01:03:37,050 And in some cases, they can do a few lines with the code by themselves. 1304 01:03:37,050 --> 01:03:39,332 If they get a little something out of it, 1305 01:03:39,332 --> 01:03:41,040 they'll get a little something out of it. 1306 01:03:41,040 --> 01:03:42,690 And they'll kind of move on. 1307 01:03:42,690 --> 01:03:45,000 And then I'll help them with the next program, as well. 1308 01:03:45,000 --> 01:03:48,360 But sometimes, it's a very little more of a direct instruction 1309 01:03:48,360 --> 01:03:49,980 that I'm kind of working with them. 1310 01:03:49,980 --> 01:03:55,300 And also, again, I will encourage them to try out some of the labs, 1311 01:03:55,300 --> 01:03:58,920 like some of the embedded videos in the program specs. 1312 01:03:58,920 --> 01:04:05,440 And eventually, they may not get to complete Vigenére or even Caesar. 1313 01:04:05,440 --> 01:04:09,010 They may just do the first few weeks of problems. 1314 01:04:09,010 --> 01:04:12,670 And then I might substitute a writing problem for one 1315 01:04:12,670 --> 01:04:15,490 of the more challenging programming problems 1316 01:04:15,490 --> 01:04:19,810 for them, so that they could have an alternate assignment, basically. 1317 01:04:19,810 --> 01:04:23,270 Keep them engaged and maybe on a topic, a current events topic or something 1318 01:04:23,270 --> 01:04:26,030 that they'd be interested in. 1319 01:04:26,030 --> 01:04:28,900 And then that will be a slightly different experience to have. 1320 01:04:28,900 --> 01:04:32,700 But hopefully, they still get something out of the class. 1321 01:04:32,700 --> 01:04:36,350 So what I was going to talk about, just some ideas here 1322 01:04:36,350 --> 01:04:39,860 that other cs50 teachers have talked about in the past 1323 01:04:39,860 --> 01:04:43,160 that I thought were really great ones, is 1324 01:04:43,160 --> 01:04:45,440 that some students, some teachers, rather, 1325 01:04:45,440 --> 01:04:50,130 will have students walk through their code to another student. 1326 01:04:50,130 --> 01:04:51,752 So I think it's a great idea. 1327 01:04:51,752 --> 01:04:53,960 It's not something I've done right now, but something 1328 01:04:53,960 --> 01:04:58,580 I would like to figure out if I could kind pair students of similar-- 1329 01:04:58,580 --> 01:05:00,913 that are working at similar paces on their programs, 1330 01:05:00,913 --> 01:05:03,830 after they complete their program, to kind of walk through and explain 1331 01:05:03,830 --> 01:05:05,803 their code to another student. 1332 01:05:05,803 --> 01:05:07,970 Some teachers will have the students actually create 1333 01:05:07,970 --> 01:05:10,280 a short video walking through their code, line 1334 01:05:10,280 --> 01:05:12,610 by line, which I think is a really interesting idea. 1335 01:05:12,610 --> 01:05:14,360 And that might be something that you could 1336 01:05:14,360 --> 01:05:16,930 do instead of having students do a written reflection, 1337 01:05:16,930 --> 01:05:22,100 kind of make sure that they're understanding what they're creating. 1338 01:05:22,100 --> 01:05:24,950 Teachers have talked about bringing back students from past years 1339 01:05:24,950 --> 01:05:28,730 to provide additional support as a TFA to the class. 1340 01:05:28,730 --> 01:05:30,270 So that's really helpful. 1341 01:05:30,270 --> 01:05:32,520 Again, I don't usually have the luxury of having that. 1342 01:05:32,520 --> 01:05:35,480 So instead I'll have student experts that 1343 01:05:35,480 --> 01:05:38,585 have completed a problem that will kind of walk around and help 1344 01:05:38,585 --> 01:05:39,710 other students with a code. 1345 01:05:39,710 --> 01:05:42,710 And a lot of students actually really enjoy that. 1346 01:05:42,710 --> 01:05:45,110 I've had students comment that they were afraid they 1347 01:05:45,110 --> 01:05:46,460 were going to be a little bored in the class 1348 01:05:46,460 --> 01:05:48,750 because they've really been programming for years. 1349 01:05:48,750 --> 01:05:53,810 But after finishing the cs50 problem, they really enjoyed helping your peers. 1350 01:05:53,810 --> 01:05:59,055 So that could be another way to run the class. 1351 01:05:59,055 --> 01:06:00,680 Again, I'll purposefully make mistakes. 1352 01:06:00,680 --> 01:06:01,990 I'll stress a lot. 1353 01:06:01,990 --> 01:06:03,740 And other teachers have talked about this, 1354 01:06:03,740 --> 01:06:07,190 as well, that they'll stress a lot the idea of making mistakes, 1355 01:06:07,190 --> 01:06:11,810 to really normalize failure, normalize errors, and make sure 1356 01:06:11,810 --> 01:06:15,917 that the classroom is a very safe space, where students feel they can't fail. 1357 01:06:15,917 --> 01:06:17,750 But that doesn't mean that they are failing. 1358 01:06:17,750 --> 01:06:20,042 It just means that they haven't quite gotten there yet. 1359 01:06:20,042 --> 01:06:21,440 That it's a process. 1360 01:06:21,440 --> 01:06:25,670 And that perseverance is really key. 1361 01:06:25,670 --> 01:06:29,060 Some teachers have talked about preventing last minute 1362 01:06:29,060 --> 01:06:33,050 panic, when they do have like little more strict deadlines for programs 1363 01:06:33,050 --> 01:06:34,270 to be due. 1364 01:06:34,270 --> 01:06:38,930 And in order to prevent that, they might have something like a 24 tardy, 1365 01:06:38,930 --> 01:06:41,630 where a student, with teacher approval, can have an extra 24 1366 01:06:41,630 --> 01:06:43,410 hours to complete a program. 1367 01:06:43,410 --> 01:06:46,160 So they're not up at 3:00 in the morning feeling like, oh, my God, 1368 01:06:46,160 --> 01:06:47,270 I'm going to fail this project. 1369 01:06:47,270 --> 01:06:48,103 I have to get it in. 1370 01:06:48,103 --> 01:06:50,480 So let me just copy from a friend. 1371 01:06:50,480 --> 01:06:54,650 So again that's something that's really worked well for other teachers. 1372 01:06:54,650 --> 01:06:57,410 Also, some teachers have had various different online discussion 1373 01:06:57,410 --> 01:07:01,430 forums, where students can, together, share their thoughts and questions 1374 01:07:01,430 --> 01:07:02,765 as they work through programs. 1375 01:07:02,765 --> 01:07:05,390 And that's something that I'm going to be looking at this year, 1376 01:07:05,390 --> 01:07:07,040 if we do start the year remote. 1377 01:07:07,040 --> 01:07:08,930 Here in New York City, we have no idea yet 1378 01:07:08,930 --> 01:07:12,330 how we're starting the school year, no idea at all. 1379 01:07:12,330 --> 01:07:15,413 So I am trying to kind of think of new ideas 1380 01:07:15,413 --> 01:07:17,330 for students to be able to collaborate and get 1381 01:07:17,330 --> 01:07:20,480 to know each other in an online forum, one that I'm moderating 1382 01:07:20,480 --> 01:07:23,600 and I'm keeping tabs on to make sure they're not sharing too much, 1383 01:07:23,600 --> 01:07:26,870 might be a really interesting way to go. 1384 01:07:26,870 --> 01:07:30,170 Some teachers also talked about white boarding and having students 1385 01:07:30,170 --> 01:07:34,070 writing on the walls or on chart paper to kind of work out solutions 1386 01:07:34,070 --> 01:07:36,450 that they then share with their classmates 1387 01:07:36,450 --> 01:07:38,580 and with the rest of the class. 1388 01:07:38,580 --> 01:07:41,360 So again, just so many different ways to run 1389 01:07:41,360 --> 01:07:44,630 the class, which is one of the reasons I love this curriculum. 1390 01:07:44,630 --> 01:07:47,750 I'm not boxed into having to do it a certain way. 1391 01:07:47,750 --> 01:07:50,840 I prefer, rather than just having students watch the lecture, 1392 01:07:50,840 --> 01:07:54,510 I do prefer to teach short segments of it myself. 1393 01:07:54,510 --> 01:07:57,650 And I think a lot of my students do enjoy that kind of teacher 1394 01:07:57,650 --> 01:08:01,350 to student relationship, in that sense. 1395 01:08:01,350 --> 01:08:04,173 But then having all these resources available for backup. 1396 01:08:04,173 --> 01:08:05,840 It's been a pleasure meeting all of you. 1397 01:08:05,840 --> 01:08:08,000 I hope everybody got something out of it. 1398 01:08:08,000 --> 01:08:10,190 I know many of you were teaching in many different environments, 1399 01:08:10,190 --> 01:08:11,482 not necessarily in high school. 1400 01:08:11,482 --> 01:08:15,210 But hopefully, it was helpful for you, as well. 1401 01:08:15,210 --> 01:08:18,500 So thank you so much for being part of this. 1402 01:08:18,500 --> 01:08:19,000