DAVID MALAN: All right. Looks like we have just about everyone back. Well, if you just joined, welcome to the CS50 Educator Workshop. My name is David Malan, and I teach this course here at Harvard in Cambridge, Massachusetts called CS50. We started these workshops over five years ago now, toward an end of just making it easier for teachers like us, like you, to adopt or adapt introductory materials like CS50 Zone. 

The course has since evolved into multiple different flavors, as you may know. Not only the version that we offer here, traditionally, on campus, but also online through Harvard's Extension School for adults. And others as a massive open online course through edX, as well as now some Follow On and some precursor courses. Some of which you might be familiar with, like CS50's Introduction to Technology, Computer Science for Business Professionals; Follow On courses on game development, web development, artificial intelligence, and more. 

So indeed, particularly at the k-12 level, or maybe 7 to 12 level, have we really been focusing in recent years, on making sure teachers have more runway. So that they can actually put together using some of their own materials and perhaps some of ours, a full three or four year program. Particularly for younger students, and now older students as well. 

So this workshop, being online, just like last summer. We're taking for granted that folks are familiar with the course in various aspects per Carter's emails over the past couple of weeks. There were materials from last summer, that we again made available this summer, just to help on-board folks do some of the basics. 

So today and tomorrow, we'll generally assume that folks are already familiar with CS50 in all of its forms to some extent. And indeed, this weekend will be as hands on and as discussion oriented as we can make it. So that you feel like, Zoom wise, it's as high value as we can do things online these days. 

Before we dive into this morning's first session, wanted to make a few introductions. I'm joined here ultimately by CS50's own Brian Yu, who you might know from the web class, the AI class, and CS50 itself. But let me first introduce you to two of our newest team members, one of whom you've heard from email already. Carter Zenke and Bernie Longboy, who have joined us now on the team here in Cambridge to focus particularly on a lot of these outreach and AP high school and older efforts. Carter, over to you. 

CARTER ZENKE: Hello, everyone. It's so wonderful to see you all in one place. I'm really excited for this workshop we have together. As David has said, and I've been saying over our emails together, this is intended to be a celebration of all of our work as educators and as teachers involved with CS50. 

To that end, today we have several presentations from educators who have deeply thought about how to make their CS50 classrooms as engaging and transformative as possible. And we'll also have some dedicated sessions for you to meet each other, and talk about your own experiences teaching CS50, and how you can further improve your practice. 

I'm really excited to see what we'll all talk about during our workshop together. Thank you all for being here. Excited to do it. 

BERNIE LONGBOY: Good morning, everyone. I just want to welcome everyone as well. And look forward to this weekend. I'm very excited with you all to learn so much too. So have a great, great morning this morning. 

DAVID MALAN: And for those who might not know him-- Brian, do you want to say a formal hello as well? 

BRIAN YU: Sure. Hi, everyone. It's so good to be here at another CS50 Educator workshop. I first attended my first Educator workshop back in 2016. And every year I've been to one, it's been amazing to see the ideas that educators have been able to bring up, and thinking about how best to support students as they learn computer science. So, it's so good to see everyone here this weekend. 

DAVID MALAN: Indeed, welcome. So within the breakout groups and smaller sessions, you'll definitely have an opportunity to meet others. Feel free to use the chat window though throughout today and tomorrow to just say hello. In fact, if you'd like to say right now, as some of you already have, "Hello", from wherever in the world you are. It's always quite fun to see just what's breadth of teachers are able now, especially that things are online, to attend with us. 

For this first morning session here in Cambridge, we thought we would offer something called "CS50 Explained, Behind the Pedagogy". In part, the goal here is to answer in one fell swoop as many frequently asked questions as we can. Particularly some that we've seen in workshops past. 

And also to resurrect something, or evoke really, something that we started doing a few years ago. Where a colleague and I would actually rewatch CA50 Zone lectures, and talk about the "Why". Students typically are more interested, daresay, in the "What", and just making sure that they're comprehending what it is we or you might be presenting in the classroom. 

But for teachers sake, we thought we would try to peel back the curtain a bit. And give you a look at, or some justification for, "why" we in CS50 do some of the things the way we do. We don't claim that these are necessarily the right ways, we can only argue that this is the reason we do things and why. But by all means, feel free to interject throughout this morning session with any questions in the chat, any virtual hands in Zoom, and certainly feel free to raise some of these same questions in later today's sessions as well. 

Brian, shall I turn the floor over to you? And Brian and I will have a conversation of sorts, even though both he and I rather know where this is going. But again, please do interject at any point. Brian? 

BRIAN YU: So I thought we would start with one of the topics that I think a lot of people ask about when they first hear about CS50 and CS50's curriculum. And that is the programming languages that CS50 ends up using. So I see on your slide that CS50 covers a lot of different languages-- Scratch, and C, and Python; SQL, HTML, CSS, and JavaScript. 

So that's a lot of different languages. Well, another reasonable approach might be for spending an entire course diving deeply into one language like just C or just Python. So I was hoping you could maybe tell us a little bit about your thinking behind choosing these languages, and multiple different languages for the class. 

DAVID MALAN: So, it's a really good question, Brian. I'm glad you brought this on up. So, I think CS50 here at Harvard has been around for some 30 years. And back when I took it myself, some 20+ years ago, I feel like I and a lot of students emerged from the class in an earlier form of it, thinking of the class as a course in C. 

And indeed, at the time, it pretty much only focused on C throughout the entire semester. And so, as a result, I and I think some of my classmates emerged, ultimately, with a mental model of "I know how to program in C" or "I took a C class". And the downside of this is, that certainly with a language like C, it's not one that you would typically reach for off-the-shelf to solve problems. 

Generally, after taking a course in it, unless you're a lower level systems type programmer-- and we, in retrospect, I wish I had been, I had appreciated as a student that what I really did learn was not just how to program in C, but to how to program more generally. But what I didn't have at the time was an opportunity to try to apply some of the lessons in that original course to just other domains. And pick up languages with the support structure of a professor and a whole teaching staff, teaching assistants, some of my undergraduate classmates. 

And so, when I took over the course some years later, and this is now 16 years ago, I had just exited graduate school. Where I had the opportunity to take a class at MIT's Media Lab, where they had literally just been beta testing Scratch, this graphical language, for the very first time. And we were using it in the graduate course I was taking. 

And so, when I finished grad school and then started teaching CS50 a year or two later, we added Scratch to the very beginning of the course. And the goal was, to send a message very early on that, one, programming doesn't need to seem or be daunting. It doesn't need to look scary textually with parentheses, and semicolons, and the like. 

Scratch, from day one, allowed us to start focusing on concepts-- on functions, variables, conditions, loops and the like without the distraction of any more textual based languages distractions. Thereafter, I have always felt that C provides a very solid foundation from the ground up. It's not without its downsides, of course, in that memory management pointers and the like tend to get very sophisticated. But insofar as CS50, both for us here on campus and for a lot of people online who've been taking it for some years, is a terminal class. 

It's the one class that they take, from which they want to get some practical skills. They want to have an ability to program, but they don't want to become computer scientists themselves. So they might not necessarily go down the rabbit hole of other courses. 

We wanted to introduce students to another procedural language or language that has procedural aspects, namely Python. Years ago, it was PHP. Now, it's Python. And Python is just wonderfully multipurpose. You can use it for command line applications, web applications, data science applications nowadays. And so it's just a nice fit for the message that we want to send. 

And then at the tail end of the class, we also want students to understand the applicability of computer science. And to programming, to just something they see and use every day in web programming. Just really does resonate, certainly with our population here, and it seems, at large. 

Unfortunately, to introduce web programming, you need this additional fire hose of content. Not just HTML and CSS to do static stuff, but you ideally want to add in a little JavaScript these days, so that you can do some things interactively in the GUI. And honestly, you ideally want some place to put data, so that you can actually build proper applications that live on beyond the course, and for that we need SQL. 

So we can see, it is a lot. And students don't become an expert in any one of these. But nowadays, I do dare say that students exit the class with a mental model of having learned how to program. Not program in all paradigms, we don't really touch much OO or functional programming, if you're familiar with those. But they absolutely learn, pretty rigorously, how to program procedurally, so to speak. Brian? 

BRIAN YU: And looking at that list, it seems like the-- maybe one that you might least expect to see on a typical introductory course is the one on C. Like you mentioned, that C's been around in CS50 since CS50 was first introduced at Harvard. And I was wondering if you could talk about why it stayed there. C is definitely, maybe, less popular now than it was before. And can you talk a little bit about the rationale behind what role C serves inside of this computer science curriculum? 

DAVID MALAN: So I do think that C is about as close to the hardware, so to speak, as you can get before things really get scary with something called Assembly language and the lower level machine instructions that are ultimately getting executed by the computer's CPU. C doesn't really have much in the way of libraries right out of the box, like a Python, like Java, even like C+ has. 

And so, if you want something to exist in C, you pretty much need to implement it yourself. Or at least there isn't a standard library in the same way as there is with other languages. That if you want a linked list, you grab it over here, if you want a hash table, you grab it over here. 

And so, it's a nice language in that syntactically or grammatically, it's relatively small. And with the couple of exceptions, students pretty much in CS50 see all syntactic features of the language, minus things like unions and function pointers. But for the most part, students then use this language, and ultimately have an understanding of what's really going on underneath the hood of the computer. 

The memory management stuff can get into the weeds for many students, but at the end of the day, you realize that all you have is this canvas of bytes. Zeros and ones, or decimal digits, however you want to think about it. And you can use those however you see fit. So as complicated as all of today's software is that we write or that we might use on our computers, it all reduces to that very simple canvas. 

And I think as a result, students exit CS50 not just thinking, hopefully, that they are budding software developers. But that they're budding engineers, and they're budding computer scientists, because they really have this bottom up understanding. 

And my favorite moment still, both in PHP and now Python, is when we go in one week having students implement with the Speller problem set, if you're familiar, their very own hash table storing words from a big dictionary. And then the next week, do we distill that same program into eight lines of Python code. One line is what gives us the hash table or the dictionary. And I think, what an amazing way to emphasize the notion of abstraction than to show students that we can take those previous several weeks and now condense them into a language feature. 

BRIAN YU: A few people in chat are mentioning that figuring out hash tables was really confusing. Yeah, I remember struggling a lot when I was first trying to implement my own hash table from scratch, working on this very assignment. But then, I also remember seeing in Python the first time, how you can do all of that in just a couple of lines of code. And really appreciating what it is that these higher level languages are also able to offer. 

And definitely at any point, I'm asking David questions. But if you have questions for me or David as well, definitely feel free to raise your virtual hand in Zoom, or type the question in the chat that I can relay to David as well. So if you have questions about CS50 and why the course does things the way that the course does things, happy to talk about those too. 

AHMAD ESSAYED: Hi, David. Hi, Brian. Hi, everyone. My question is about the shorts. Sometimes the shorts, I feel they-- sometimes they're helpful, but sometimes they are adding more confusion. So, sometimes David is covering one topic, but this topic needs more information. And David is leaving that to the shorts to cover. But I have the feeling that students sometimes need this to be covered in the lecture itself by David. 

So what's the rationale behind shorts, why not do it all in the lecture? And some of the shorts, they actually need to be discussed before the lectures. Before the lecture. Some of them, they need to be discussed after the lecture. And there is no roadmap for students. So they assume that they watch the lecture, then the shorts. But some of the topics in the shorts-- I feel that they should have been covered before the lecture itself. So-- 

DAVID MALAN: Interesting. In the chat, Ahmad, do you mind proposing a few examples? But in the meantime, I can respond in general. So, at least the way the traditional class here is structured on campus, we have certain constraints like time. Like the duration that a lecture can be. 

And the function of lectures, generally in our view, is to create this narrative for the week. To introduce students to a range of concepts that will then be explored in some form on the problem set. But we don't have the luxury, just practically speaking, to go down every rabbit hole. Or to focus as much as we might like in an ideal world on each of those topics in depth during that one seeding. 

So the approach we've taken over time, is to then define these shorts. Which, if unfamiliar, are indeed these shorter videos generally taught by a colleague of ours, Doug Lloyd, against a nice white backdrop. Wherein he focuses entirely on one topic-- so this is a topic on linked list, this is a short on arrays, this is a short on some other topic. 

And the goal there is to, one, indeed go down the rabbit hole of those individual topics. And also, frankly, pedagogically, to give students another person's perspective on the same topic. It's not just a rehash of the exact same examples, the exact same style. It's Doug's own take on that same material. Much like a student in our on-campus sections, or recitations, would see different exposure. 

I think, pragmatically speaking, if you're finding that certain shorts should come before a lecture, please do mention as much in the chat, or email me and Brian afterward. Because that's an easy fix. We can certainly guide students toward that more correctly. But that's not the intent. Generally, they're meant to be supplementary, optional resources for students if they want to dive in deeper to topics we looked at that week. 

Brian, if you don't mind my turning the tables here, because you've given quite a bit of thought to how we grade in CS50. And we grade in different ways based on the demographic of students, whether they're the traditional on-campus students, or the online students, or certainly via the massive open online course. It's a different experience to some extent for the students, as to just how much feedback we can viably provide tens of thousands of students online. 

So, Brian, we've generally presented to teachers in the past, that we take this three pronged approach. Three axis of correctness, design, and style. Do you want to speak to exactly what we mean by those three, and why we use those three? 

BRIAN YU: So these are the three things that we generally ask students to think about when they're working on their assignments. And the first one, and arguably the most important one, is just correctness. Which in our mind of answering the question-- does the student's code do, what our problem specification asked for the student's program to do? If we're asking them to build a pyramid made out of hash marks and spaces, are they using the right arrangement of hash marks and spaces? If they're building a spell checker, does it correctly identify the words that are misspelled? 

And it used to be the case that for correctness, I would manually test and run each of my students' programs, and test it on various different inputs. But now, we've built this tool called check50, which can be used by students and educators alike to do some automated assessment of correctness. And that's really helped to improve the speed at which we can get some feedback back to students. 

So students can be working on their own time, working on a problem, and run this tool, check50, to see-- have I really managed to solve this problem? Or are there maybe some edge cases that I haven't considered, or some bug in the program that I haven't quite noticed. And check50 can help students to think about the correctness of their program there as well. 

But we don't really want students to just stop there. That there are many ways to write a correct program that solves the problem. But some ways are going to be better than other ways. Better in terms of the efficiency and how long the program takes to run. Better in terms of the overall organization of the code and whether it's logical and makes sense, and other considerations that we generally categorize under the second heading of design. 

And design, then, is all about how well written is the code. Even if regardless of whether it's correct or not. Are they making good use of programming constructs like loops and conditions where it's appropriate to do so. Are they repeating themselves a lot when it might be better to create a different function to do some of that same behavior. 

And so, here is where really the opportunity is to teach good design practices. Teaching not only how to write code, but how to write code well. And then the final consideration that we look at when grading is style. 

And style is less about the efficiency of the code, and is more just about how the code looks. Because, especially if students are aspiring to become software engineers or working in computer science in the future-- writing code that is readable by other people, whether it's by coworkers, or other people on the internet, or even you as the educator, that's really important too. 

So that it's clear what you mean when you're writing code. That it's easy to follow and easy to read. And we want to encourage those best style practices too. 

DAVID MALAN: Thank you, Brian. There was a question in the chat about a project, but would be best to email that to someone, something like certificates@cs50.harvard.edu. We can paste that email into the chat. Well, transitioning perhaps to-- no, actually, one note on grading philosophy if I may. 

So here on campus, where we have the luxury of having a reasonably manageable ratio between teachers and students by nature of it being a traditional course offering, here in a college environment where we have a number of TAs. And so, we're able to provide feedback along each of these axis. 

Generally speaking our online students, whether taking CS50 AP through edX, or CS50x, or any of the courses, do indeed as Brian notes, rely on the automated correctness tool, namely check50 in our case, to focus on correctness. Style too, to some extent, we rely on style50-- a linter tool that provides, indeed, this sort of quantitative feedback as best it can. 

Design though, is really the most labor intensive axis along which to evaluate students' work. And certainly teachers in high school, if I think back when I had some 100 students. And I was getting homework per week, and that was just in math not even in computer science. Is it pretty daunting to even imagine going through and qualitatively providing feedback as well? 

So these are among the knobs that we generally find that teachers turn. Focusing, for instance, generally on only one, maybe two, of these axis, but not necessarily all three. Fortunately, with the trends being what they are in artificial intelligence and machine learning, we're hopeful that even design will be able to be automated to some extent by computers before long. But at least within our ecosystem, we're not quite there yet. 

But thought we would now pivot to what we've described, and what most anyone would describe, as memorable moments, hopefully. Particularly among students, seeing what they see-- on our case on camera, but what you probably do most any day in the classroom, any time you try to bring some topic to life. And we thought we would give just a few examples of some of the visuals, some of the theatrics that we try to integrate into CS50 here itself. 

And, by way of a few photos, might you recognize this? These are getting harder and harder to find, honestly. But if you ever want to do a demonstration of binary search, or divide and conquer more generally, or really problem solving most generally-- I definitely recommend keeping an eye out for these in the mail room or on someone's shelf who clearly doesn't need that phone book anymore. And to use this, as perhaps one of the earliest visuals to present to students. 

And nowadays, I think it's probably important to link this physical device to the Virtual Contacts app in everyone's Android device or iPhone device. But it really is the same problem. So even though this is the old school incarnation of it, this ripping of the phone book at the beginning of CS50, which probably is not something students often see, is one of the memorable moments that we try to create. 

So that even as we get into the weeds of these fairly arcane topics, very unfamiliar topics for many students, they can latch onto, "OK, binary search. Oh, phone book." And they at least have those visuals that can help remind them of the topic. And at least help them find that same material in their mind's eye or, say, on the course's website. 

Brian, this next one here is representative of a general problem solving technique. Do you want to speak to why we put this black box on the stage? 

BRIAN YU: So we use the black box to introduce a topic in computer science that's commonly known as abstraction. And having worked with CS50 students year after year, this is one of the trickiest topics for students to really understand at first. What does abstraction mean? 

And one of the ways that we try to frame it, and there are lots of different ways of trying to frame abstraction and what it's all about, is in recognizing that dealing with all of the details of every part of a computer program might get extremely complex and difficult to reason about. But if you can start to think about things as more of a black box, you provide input into some part of the program, and that black box might produce some output. Then thinking about those components of the program, even if you don't understand everything going on inside of that black box, can help you to build bigger pieces together and construct more interesting programs. 

And that's a tricky concept to understand and code at first. And we hope that the black box visual is a bit of a metaphor that students can use for thinking about that type of thing too. 

DAVID MALAN: Brian, let me pluck off a couple of questions from the chat. Can CS50x students have access to sections as recorded in video? So this is actually one of the reasons too, why we provide the shorts, as well as most recently, the videos within the labs that were introduced this past year. Because those sections are generally more discussion oriented, and not high information content for the viewer. And so, those two alternative exists. 

Stephen notes that his Zoomer students don't know what a phone book is. Indeed, this is the reality that we're bumping up against, and the mod proposes using a hardcover dictionary. I've not been able to get myself to get comfortable with the idea of tearing actual books really meant to, or destined to be recycled anyway. So I would, perhaps, beware tearing in half something that might have more impact than something discardable, annually, like a dictionary. 

So this is one that took a colleague of ours some time to build. And it doesn't need to be this sophisticated, and indeed, you're about to see a few photos that are the result of our having collaborated this past year with a wonderfully talented prop shop as well as colleagues across campus. Where we've had the resources and some of the hardware around with which to build some of these things. 

But all of these are born of much simpler ideas. Before we had some metal and eight light bulbs and electricity like this, I would just have a few students, three students, eight students come to the front of the room and hold up their phones, which generally have flashlights these days. Or I picked up a few desk lamps for a few dollars at Target, a local retail store. 

So as to represent what we've called here, binary bulbs. Light bulbs that represent bits or transistors being turned on and off. And metaphorically being the light bulb, we hope, goes off in students' head very early in the semester. And if you've not seen this one before, these are magnetic numbers that a child might play with on the refrigerator, that just represent what place each of these light bulbs represents. The lowest ordered bit on the right, and the highest order bit on the left. 

We use this to just have some visual fun talking about binary. It used turning switches on and off here, but again, any light bulb suffices. And then we dramatically, because of the magnets, have a second student come up typically during this demonstration. And I then try to dramatically wipe off all of the magnetic letters, such that they fall on the floor. And so now, it's like Luke Skywalker mode, trying to visualize only in their mind's eye what each place represents, so as to ramp things up. 

But here too, even though we might end up spending a few extra minutes on a demonstration like this, it's a lot more interesting certainly than going to a chalkboard or a whiteboard and just jotting down patterns of zeros and ones. Again, trying to bring the topic to life is the goal of a visual like this. Whether it's complicated as this build, or as simple as just having students use their own phones and flashlights. 

So this is a larger duck than you might have on your desk at home. And this is deliberately meant to be, again, another dramatic visual that is actually incarnated in smaller form. But Brian, do you want to speak to why there's such a big duck here, or rather a smaller duck that we tend to hand out to students back in healthier times here on campus? 

BRIAN YU: This is one of my favorites. This little thing is a duck debugger. And one of the most common things you have to teach students when they're first working with computer science is, how do you debug their code? Odds are, their programs won't work on the first try. And there are all sorts of techniques for debugging. Adding print statements to see what's going on, we teach students how to use a graphical debugger where they can step through their code line by line and look at the values of variables. 

But maybe one of the simplest is just using one of these little rubber ducks and using a technique known as "rubber duck debugging". Where the student just talks to the duck, and explains to the duck, what their code is doing and what they're trying to get their code to do. And amazingly, even though I wouldn't have thought so at first, just talking through the problem to the duck is often enough for you to realize, "Oh, this is exactly what my mistake is." 

The act of articulating the problem trying to explain it from first principles helps you to realize what's gone wrong. And ultimately, eventually you don't even need the duck. But especially when you're starting out, just having something there as a reminder of something you can talk to as you're working through your problems can definitely be quite helpful. 

And even in CS50 IDE, the online code editor that we have students use in CS50, there's a little virtual rubber duck in the upper right hand corner. So that students can use that as a debugging technique, if they'd like to try working on debugging their code. 

DAVID MALAN: And this is the sort of thing that, it's not free. And we actually stamp our own name CS50 on the rubber ducks that we find online. But I'm actually in a few teacher Facebook groups, where every year it's fun to see teachers posting online some deal they found on Amazon for just a few cents each. Getting a whole bunch, like a bag of these rubber ducks on sale to present to their students. 

So if resources permit, it's a fun way to, again, bring to life some topic and also genuinely empower students with a technique that might very well help them. And frankly, it doesn't need to be a duck. It can be anything at all. It can be a picture of a duck that they draw themselves. So even there, too, there's a range of possibilities that doesn't need to be a physical thing as well. 

So here, this has been a dream of mine for 15+ years to actually talk about binary search. Not just with the phone book, but with actual doors. Since I've long envisioned this idea, this theatricality of being able to walk in front of a bunch of doors, open the door, and look behind it to see what number is hidden there. Toward an end of introducing students to the problem, more generally, of search. 

For instance, searching an array of numbers, and array being applicable to our focus on C early on in the semester. And by nature of COVID this past year, unfortunately, not only was most everything shut down, so was one of the local theaters here in Cambridge. As the result of which was, that we did have this rare opportunity to collaborate with their own prop shop. 

And so, what you're seeing here is a photograph of eight or so doors that were just in storage at the back of the theater that they were able to bring back to the front of the stage. So as to do a demonstration like this. And this is not one that, generally, we can do here in healthy times. Most of us probably don't have access to eight full fledged, human sized doors. 

However, those of you who are in high schools or middle schools probably have a hallway full of lockers or a gymnasium that has something like that. And so, what we've also done in other times is actually get a few metal lockers, either on campus or bring them into the space temporarily. And being able to use that physical demonstration is nice in that, when talking about arrays, when talking about searching behind those arrays, the fact that there is this physical door-- especially in the world of C or really any language, Java included, that supports arrays, makes all the clearer that a computer can really only look at one of these values at a time. 

The problem, of course, being if you just put a whole bunch of numbers on the stage or in the front of the classroom, yes, you can say to students that they should only look at one number at a time when searching for a particular value. But as humans, we all probably take in everything we're seeing all at once, and boom, there's the number you're looking for. Or boom, it's over there. 

There's not necessarily the opportunity to do the mechanics of searching from left to right, or from right to left, or even binary search in the middle on down. So using something that physically closes like doors in this one case, and sadly we won't have access to these anymore. But-- or lockers, or even paper cups, or something which you can use to hide something there or not, we found to be of another very effective visual for students to latch on to. 

Brian, meanwhile, did have an opportunity to use some actual numbers. This time, we didn't use the doors because we thought it would be way too much opening and closing. And at this point, we wanted to talk to students about sorting those very same numbers. 

And if you're given a bunch of numbers like these in random order, these happen to be plastic numbers that we found on Amazon, they even have little light bulbs if you put in a tiny battery that turn on and off. But we just wanted some physical numbers. And in the past, we've done this with sheets of paper, where we just write the numbers on the board. Here we wanted to try something a little more physical. And Brian, do you to explain what you did with these numbers? 

BRIAN YU: So the idea of using these numbers was to visualize what it looks like to perform a sorting algorithm, something like bubble sort, or selections for example. And the nice thing about using physical numbers on a shelf is that it reminds students of a couple of important properties of what it's really like to sort numbers in an array. 

One being, that there's not a whole lot of shelf space here. So it wouldn't be that easy to take a number, and squeeze it in between two other numbers without making the space tight. So if you want to move numbers around, you really need to take one number, remove it from its place, take another number, and put it there, for example, to really emphasize that each of these numbers is in some particular spot inside of an array. And you can remove something from it, in order to make room for something else. 

And the other nice property is that these numbers are, you have to hold them in order to move them around. And it's not easy for me, just to pick up all eight of the numbers, and then put them exactly where I want. I can really only be working with one or maybe two numbers at any given time. 

Which is a good reminder to students as well that the program can really be working with-- your program is doing one thing at any given time. And it's taking a variable, putting some value inside of it, swapping some value or something like that. And so, this offers a good way of visualizing what these algorithms look like, seeing them a little bit more slowly, step by step. And I think it's helpful for understanding how it is that these algorithms work. 

DAVID MALAN: For those who missed it in the chat, shoe boxes were proposed too, as a mechanism for hiding some value that might be inside. I like that one as well. Now in healthier times, this is a photograph of some students we actually had come up on stage. Brian was doing that in isolation this past year. 

But when we have the opportunity to have students come to the front of the class, we have in some semesters just handed them those physical numbers. Either the plastic numbers here, or just print outs in very bold fonts. So that it's clear to the students sitting down, what numbers the students are holding. 

And here too, it's a nice visual in that there's a lot of movement. And the fact that it is so expensive to keep walking back and forth, moving these numbers around, rather sets in. The downside of using humans and students for these kinds of demonstrations, this is, that I do think it's also a little harder for the audience. Especially if they're not 100% following along with this new topic, like bubble sort, or selection, or the like to understand what's going on, because it's a little sloppy. 

And so, I would keep that in mind any time you do use human participants. And that the demonstration might not very well go according to plan. And that's fine too, if it's just meant to be a memorable moment that you can then come back to again and again. 

But there's something to be said too, I think, for the cleanliness of doing it yourself, or as Brian did it, alone. So that you can really choreograph things and get it just right. But there's perhaps a nice balance between those two. 

Some of you might have seen the videos online of, I think it's a Hungarian dance group does something for bubble sort, and selection sort, and insertion sort and others. And those are fun too, but they're rather for the viewer that already knows what's going on. Because there's a lot more spinning than there needs to be, for instance, in a typical sorting algorithm. 

So this is a fun one that can happen very early on in the semester, whether using C or any language, when it comes time to just talk about variables, and assigning values to variables, and swapping values. And we introduced this a few weeks in, if only because a lot of these algorithms we just talked about-- selection sort, bubble sort, and the like, generally involves swapping the locations of two things in memory. 

Now in Python, you can do this with one line of code that just magically swaps two things. In most languages, you need to be a little more pedantic unless you resort to bitwise operators and the like. And you need generally a temporary variable. And so, Brian, when you did this, this year, what was your goal? 

BRIAN YU: So the question we asked the students is, how would you change which glass contains which liquid? Like get the glass on the left to contain the blue liquid instead. And get the glass on the right to contain the red liquid. And most of the students were able to pretty quickly come up with the realization that if you just have these two glasses, you can't quite do that. 

And that the solution is to use a third glass. You pour the red liquid maybe into the third empty glass. And now, you have room to take the blue liquid, put it into the glass that used to hold the red liquid, and then transfer the red liquid into the new glass as well. 

And this is always a great demonstration because if you just gave students two variables, an integer called X and an integer called Y, and told them to swap them, most won't immediately come up with exactly what the right lines of code are to perform that swapping procedure. But introducing this demo first, most students figure out the logic very quickly of being able to see the actual liquid, seeing how they need to be poured into other glasses to get them to swap, and then you can translate that back into what the original C code looks like. 

DAVID MALAN: And when it's an actual student doing this, I mean, we deliberately only give them two glasses. We ask them to swap the two values. And then, there's the sort of awkward moment where they realize this is not really going to be easy or even possible. At which point, you can whip out the temporary variable, the third glass in the story. When it's, for instance, the student and not just the teacher or Brian. 

Just to now give you a taste of some of the other visuals we've used in recent years. We found an old spare mailbox that we've used as a metaphor to talk about addresses in the context of C, and how every location in memory indeed has an address of some sort. 

We use this as an opportunity to talk briefly about hexadecimal, if only because they're going to see zero X dot dot dot in a bunch of context like the debugger, or perhaps other tools. And that's a fun metaphor. And even if this isn't something to bring to the front of the classroom, you could imagine doing a little skit on video, even in advance, using one's own neighborhood and mailboxes along the street. 

We do use a dictionary toward the end of the semester. Not for the tearing example, but to talk about dictionaries with key value pairs and hash tables underneath the hood. And this, honestly, doesn't serve much more function other than, "hey all, here's a dictionary." This is what we're talking about. Just again, to put a visual on the stage or in the front of the classroom so that students again have like a visual to latch onto when thinking about or revisiting an otherwise arcane topic. 

This, too, was a side effect of having access to a whole prop shop. We, for the first time in 15 years, had access to an actual refrigerator that we were able to bring out to the front of the class. And we use this to talk about locking and race conditions in the context of SQL. Long story, short, there are certain challenges in computing whenever you have multiple threads or multiple users trying to do things at roughly the same time. 

Bad things can happen if two different users, two different threads are trying to access the same value. And the way I learned about race conditions back in the day is in the context of a refrigerator and opening it up, checking to see if you have some milk in the fridge. And if not, you walk off to the store, and you go and buy some milk. 

But problem arises if another family member or roommate follows you and opens the door after you've left, sees that there's no milk, and then tries to go to the store as well. And then, you end up with twice as much milk. So in this case, we actually, and you can just see it on the floor there. 

We actually locked the thing physically with the padlock and a chain just to bring to life that very low level topic for students as well. This could certainly be achieved with a camera in your own kitchen, just to make that same point. 

We talk about cookies. This is just an excuse to have cookies for the team toward the end of class here. But when talking about cookies in the context of HTTP and a shopping cart, turns out if you have a local theater, they might very well have an old shopping cart that we used to talk about state. And to talk about how cookies can be used to create the illusion of sessions, the maintenance of state across an otherwise stateless protocol like HTTP. 

So again, just another memorable moment that we tried to create. And then this one too, is meant to be, this happens to be an old phone but you could do it with any phone in the context of JavaScript asynchronous functions and callbacks. Whereby you call the function, and you want it to call another function when it has your return value essentially. 

Brian and I have started doing a little skit in the past couple of years. Where I call him on stage, and then I ask him a question that he needs a little bit of time to get the answer to. So we hang up, I continue the class, and then a few seconds, minutes later, the phone rings with my callback. At which point, he has the answer to my question. So a fun little skit too, that can be done certainly with a couple of cell phones in one's pockets. 

SPEAKER 1: In this pandemic, where we have to work from home and teach for home, have you used some digital tools to substitute to this physical one? To exemplify that your show? 

DAVID MALAN: It's a good question. I did not. I had the fortune to be able to continue teaching on campus this fall. No students were here, but we were in fact in this space. But let me invite the group, if most everyone here was indeed teaching online or hybrid in some form this year, please do feel free to volunteer via the chat-- inspirations you might have had using your own refrigerator, or glasses in the cupboard, or anything along those lines. Since all of us did have to get creative in some form. 

And now, as for the course's problem sets. Wanted to touch on a couple of final notes here this morning to give folks a sense of the intent of the problem sets. Which ultimately are meant to be very domain inspired. They're meant to be more puzzle like than they are meant to be the more traditional end of a chapter in a textbook problems. That just has you implement a very small, very well defined problem like implement a function that takes two arguments and return the sum of those two. 

Useful practice and good reinforcement for muscle memory, but generally speaking for CS50's own problem sets or weekly projects, the intent for us has long been to take some inspiration from a real world problem. And even if at the end of the day, the tools students need to solve it are pretty mundane like use a loop, or use some variables, or some other feature of the language. 

We want them to approach the problem with a much more alluring packaging to it. A much more alluring context. So that they actually see the forest for the trees. And Brian, do you want to just rattle off some of the domains that we've used successfully here? 

BRIAN YU: One of the big goals has been to, since many students have never really seen computer science before, to really emphasize how much computer science can be connected to whatever it is that students are already interested in. So if they're already interested in games, there's a very early problem set where students are building a pyramid as though it were from the Super Mario Bros video game. And so, they see the connection there. 

For students that are maybe interested in science, we've recently introduced more science based problem sets where they're looking at DNA sequences, and DNA sequences can really just be treated as a string of text. And so, all of the opportunities for working with problems that are manipulating strings of text, we can then apply to those sorts of biology inspired problems. 

We can also apply them to problems inspired by literature, taking examples from books from various different grade levels, and assessing how difficult those books might be to read all programmatically. And we've taken inspiration from software that students are using, apps that they're often using on their phone. 

Especially the filters that apps like Instagram or Snapchat might be quite popular nowadays. And so we ask students when we introduce files and images, to try building some of those filters for themselves. Writing filters to apply various different changes to existing images too. 

We explore the world of cryptography and figuring out how to encrypt and decrypt information in order to send secret messages. And ultimately, just trying to come up with these sorts of problems that students will see the practical value of it, and hopefully be excited by it. So that the problem domain is interesting, and they're therefore able to really get engaged with the code that they're writing to. 

DAVID MALAN: And for what it's worth, the approach we've taken over the years to finding problems like this, is just to keep a passive eye and ear out over time. When we see some interesting blog post online about some new technology, we start to think about how to distill that seemingly complicated technology into its essence. For instance, like file recovery in the forensics example that Brian gave. 

Or if we aren't ourselves familiar with some field, for instance, the extent of my background in biology is from ninth grade in high school and that's it. Thankfully, Brian has a little bit more of a background and was able to create our more genetically inspired problems most recently. So that, even we are learning something along the way. 

The upside of which is, that we need to wrap our minds around a complicated topic and then again distill it into something simplified for students. But just so that they start to see the connections of computer science and programming to these other worlds. 

As an aside, we see a whole bunch of questions that have been flying across the screen. We're taking notes on all of the chat questions. So even if we don't touch them all live here, we'll follow up in some form, so as to not leave those unaddressed. 

So in closing here, there's a few things we wanted to touch on as well as to the events that we've done here. At least in healthier times, CS50 has been punctuated at the beginning, the middle, and the end of the semester by way of events that we've brought online over time. We didn't start the course this way, but each time over the years, did we try to introduce a little something new to see what sticks and resonates with students. 

For instance, years ago, we wanted to send the message very early on in the course, before it begins practically, that computer science is not about programming per se. And even though that's the mental model that many students have, particularly given how it's structured in lower grade levels, that it's programming-- we want them to there see that it's not just what their friends have been doing, middle school or high school, it's really about problem solving more generally. 

And indeed, once you get to college, at least in our case, are there are just so many other directions that you can typically go in. And we want to give students a taste of those various directions, but not by focusing on code and on programming per se, but on taking input and producing output. Ergo solving problems. 

So CS50 Puzzle Day is something we've done for years with some alumni and friends at Facebook. We're really good at writing these riddles and puzzles, mental puzzles not physical ones. And we print out a PDF of puzzles at the start of the semester, we typically gather a few hundred students in our case. And then, we make these same puzzles available to anyone online, taking or teaching CS50x or CS50 AP. 

So as to have a lot of these same events localized in a local school, non-profit, or community, or the like. To really just get students excited at the very beginning of the experience about solving problems. Toward the end of the semester, have we typically done a CS50 hackathon. This too, is inspired by a tradition that rather began in tech it would seem. 

But we gather in our case at 7:00 PM, and stay until 7:00 AM, working throughout the night, which is not a time frame that works well for all grade levels. So we've also adapted this idea over the years, we held a hackathon in Manhattan a few years ago with a few public and a few private schools for an AP audience. 

Focusing in that demographic on, I think, 10 AM to 3 PM. And here I was thinking that 10 AM was the ungodly hour on the weekend. But we adapted for the different demographics as well. 

And so, this is an opportunity in our case to either have students focus on one of the course's problem sets, or some other programming project, or their own final project depending on where in the school year or the semester it falls. And it's just an opportunity to normalize and socialize, what is effectively homework. But in a way that makes it feel like a shared experience, and hopefully a memorable moment in their middle school, high school or in our case, college career. 

And then lastly, is the CS50 fair. And Brian, do you want to speak to exactly what happens at the CS50 fair? Whether on a small scale with one class, or a large scale on campus? 

BRIAN YU: This is maybe my favorite of the events. CS50 students typically at the end of the class work on a final project of their own choosing. Where they get to decide on something they're interested in, and have the opportunity to showcase it to others at the fair. 

And so, we usually set up some tables where students just bring their computer. And we invite other teachers and students from around the university to come visit, and students from other schools as well. Just to come walk around and talk to the students and see what projects that they've created. 

And it's a really great opportunity for the students to have the chance to showcase what they've been working so hard on. And it's always inspiring as a spectator, just to go around and see all of the amazing work that students have been doing. 

DAVID MALAN: Well, I want to be mindful of the time here. So in a moment, let me turn things back over to Carter and to Bernie to give you a sense of what more comes today and tomorrow. Brian and I and the team will be popping out and in of the sessions today and tomorrow. And by all means, always reach out via email this weekend or beyond if you have questions. And for the chat questions we didn't get to hear during the hour, we will follow up on as many of those as we can via email after this. But thank you, and Carter, back to you.