SHAANAN COHNEY: OK, hello, everyone.
So because I've always wanted to do this,
and I do it in my own classroom, lights, camera, and action.
[MUSIC PLAYING]
[BEGIN VIDEO PLAYBACK]
[PHONE RINGING]
- Hello?
Oh my god.
FOA is starting?
OK, I need my puffer.
Where's my puffer?
- I think ChatGPT can be used effectively.
[PHONE RINGING]
- Oh my gosh.
Sorry.
FOA is starting now?
Bye bye.
[PHONE RINGING]
- It's time for Foundations of Algorithms?
I need to get my sonic screwdriver!
- I'm really looking forward to my military coup of the university.
- Me too.
I can't wait to get rid of these hot desks.
[PHONE RINGING]
- Oh my god!
FOA is starting!
- What?
FOA is starting?
I should go get my parachute!
- I've got to set my cryptowatch!
[PHONE RINGING]
- Wait, it's time FOA already?
Let me get my cat.
[PHONE RINGING]
- FOA is starting?
I've got to go meet the teaching fellows.
[END PLAYBACK]
SHAANAN COHNEY: So welcome, all of you, to today's class,
which is Foundations of Algorithms with me, Dr. Shaanan Cohney, here
at the University of Melbourne.
So for the next few minutes, all of you are now my students.
And if I look out over there into the classroom, I can see about 450 of you
this semester.
And were we in my other semester, there would be somewhere around 1,000 of you
out there in the auditorium.
All of you come from a variety of degree programs.
In my university, in Australia, there are six degree programs
that we're anticipating that all of you come
from, from places as wide as music, commerce, and the business school
to bio sciences and, of course, the faculty of science that many of you
are coming to us from.
We have 10 teaching fellows, also known as teaching assistants,
or TAs in other countries, who are going to be helping out with our course.
And then four first-year fellows, that I'll talk a little bit more about
later, who help answer some of the questions
that you'll give us throughout the semester.
We have one production associate who is one of our very own students
and is going to be helping out with the camera.
Hello, Hrithik over there.
I mean, he's back in Australia, so he probably
can't see me all the way from Boston, but I'll give him a wave anyway.
And then, of course, we have our two professors
who are going to be delivering at least the lecture components of the class.
There is Dr. Jianzhong Qi on my left at least, and of course, myself.
So today, I'm going to tell you about the 80% outcome with 5% resources
method, or what I call the 80 in 5 that I, Shaanan Cohney,
have applied to my class, Foundations of Algorithms,
that is quite similar to Harvard CS50.
And how I tried to reverse engineer some of the components of CS50
and adapt them to my own university context.
So a tiny bit about me.
Now, we're back in the Boston, Cambridge, Harvard
room, where I can now speak to you as stepping outside of our classroom
and back into the educators' workshop.
So I am responsible for the first-year computer science curriculum
at the University of Melbourne.
We have about 1,000 to 1,500 students who take the introductory course with us
every year.
And we have about 5,000 overall students in a university with about 40,000
students total.
So a very significant fraction of the overall student body
takes our class, just like the students who take Harvard CS50 in the college
here.
So the main theme of today's talk is going
to be thinking about where the biggest room for improvement is in teaching.
And while I've spent a lot of time reflecting on this,
I want you as well to keep in mind today,
where do you think you could make a small improvement that
would be a large change in terms of the actual outcome?
So is there a 5% change that you could make that would
result in an 80% different outcome?
And that's really today's talk.
So as much as we're going to draw very heavily from CS50's curriculum
structure, pedagogy, and design, at the end
of the day, whatever context you're teaching in
is going to be a little bit different.
And so you're going to have to identify your own set of changes
that you can make.
Some of them will be a little bit smaller.
Some of them will be a bit larger.
But whatever your resources are, you've got
to figure out the right things to do that will have the largest changes.
So we already know a little bit about CS50 from all the discussion we've had
so far.
But what I teach is really a bit more like CS50 plus 1/2.
So it's not the very, very intro class.
They've taken a little bit of programming before in Python.
And now, they're moving on to something slightly more difficult.
And, of course, because I'm operating in a different university framework
and particularly in higher education, I have
to fit within the structure of the larger university
and can't just teach CS50 directly without some longer term changes.
So CS50, as we know, is a lot of resources, plenty of staff,
administrative support, certain financial support.
And also from a practical perspective, there's
a lot that actually makes it work the way it's taught here at Harvard College.
They also have a lot of TAs and just like me, a very large enrollment.
However, unlike me, they also have to cater to all these very
substantial external actors.
And so taking that aside for a minute, catering just to my classroom
or just to my university, it gives me a little bit more flexibility
around what I do.
I don't have to worry about my 1.4 million YouTube subscribers.
So what does my class look like as opposed to what CS50 is like?
What changes do I have to make?
Well, firstly, we've only got two professors,
and we're very, very part-time.
So I spend half of one semester I can dedicate to this class.
And it's only about 0.2 of a full-time position for each of us.
This equates to 400 hours of work per semester,
give or take, which is 36 hours of lectures, where we're actually
standing in front of the class.
And then we also do have a little bit of TA time, but not very much.
So beyond the sections that we run every week, we are given two hours of TA
time per student that's enrolled to make sure
that we get all the grading done and all the additional help
that that student might need.
There are no other resources that the university provides.
So everything that we have to do, we have to do on this smaller budget.
And this is really important because almost none of us are here at CS50.
I'm very lucky to be here today, but the rest of us are in our own scenarios
where our resources are going to look different.
And so whatever your resources are, you're
going to have to think, OK, how do I spend my resources to get the best
bang for buck?
What are the things that CS50 does that are really effective, that you don't
need a large budget for, that you don't need a giant staff for,
that you can do in your own classroom?
So here's where we take seriously one half of CS50's invitation.
So CS50's invitation is to adopt, but it's also to adapt.
This means that you don't have to necessarily take the whole course as is.
You can take the pieces that work very well for your setting.
These pieces might be the lectures, but they
might be some of the pedagogy, some of the class policies.
Maybe it's the way they handle extensions.
Maybe it's some of the tooling.
All of these different things are components
that you could consider incorporating to your class,
whether you're teaching CS50 itself or something a little different,
like the class that I teach.
So our theme here is, can we get 80% of what
CS50 is doing in each of these different realms with only, let's say,
5% of the inputs--
5% of the staff time, 5% of the finances, 5% of the practical support?
So what makes CS50 work?
What are the different pieces that we could potentially take?
You've heard a lot, particularly in Guy's talk yesterday,
he spoke a lot about what makes CS50 tick.
And this is my version of the different pieces
that I think you could potentially select from.
And we'll go through these one by one and talk about the pieces
that I was able to take for my class and how they were effective,
even in a lower resourced environment.
So we'll start off with curriculum, and we'll
talk first about where my curriculum began
when I joined the university adopting a class that already existed.
And then where we've ended up today.
So the CS50 curriculum looks something like this--
a week of very basic introduction to programming in Scratch,
followed by some substantial work in C, talking a bit about algorithms
in that component.
We all remember the Tiedeman assignment.
Then a week to get us ready to go to Python
and then three weeks that really take the skills that students
have learned earlier and putting them in a setting where
students can build a real web app.
So there are three weeks dedicated to learning SQL, Flask, and then
the project that allows students to really take
that foundational knowledge they've used before
and demonstrate it in a very broad and flexible way.
The Melbourne curriculum before I joined-- so I joined in 2021--
looked a little similar, but also a little different.
So there was six weeks of Introduction to programming in C
and there was a lot of discussion of algorithms.
And then the final six weeks of the class in our 12-week semester was spent
dedicated to various different types of data structures and algorithms,
which is appropriate given that this wasn't their very first class.
They'd already had some exposure to programming before.
So the next six weeks spent a little time going through some basic data
structures and basic algorithms.
But CS50 seems to cover more in less time during the semester
than we were covering in a full 12 weeks.
So in just 10 weeks, CS50 is cramming in a lot of content.
It covers a lot of the algorithm's content
that we did, perhaps not quite as far, but then
it also managed to get all the way through teaching Python,
substantial amounts of web development, and all that in really
just 10 weeks of lectures.
So how is that done?
Well, you could say all we need to do is we
should take exactly the things that CS50 teaches and then put them
in our classroom, and everything will instantly be fixed.
Unfortunately, the reason that CS50 works is not just because of the things
they've chosen to teach.
And so it's a mistake to adopt someone else's content whole scale
without understanding how it's delivered and why it works in their context.
So you also have to think about the other components of CS50
that make teaching this class in 10 weeks a viable proposition.
So this includes the pedagogy, the way they think about how they're teaching
and how things are structured, the quality of the delivery,
and the excitement of the delivery that engages students and gets
them to put in that necessary effort, as well as the assessment components, which
is really where students do a lot of their self-driven learning.
And ultimately, where they're going to pick up the skills that they see
demonstrated in lecture.
So if we don't have students putting in the same amount of effort and time,
we're probably not going to be able to cover the same content in 10 weeks
because students will fall behind.
So now that we've taken at least a first look at curriculum,
and we'll come back a bit later to where we landed, we should talk about some
of these other ingredients that will enable us to make this curricular shift.
So one thing that students often say about CS50,
and something that I think I've managed to accomplish in my classroom,
is to make the class feel a bit like a movie.
So we've all watched the CS50 lectures.
We see how it's presented in super high production quality that's filmed in 4K,
ultra high definition, high dynamic range,
all these little buzzwords that end up giving you a good picture and good sound
quality.
And particularly with the sound, students
notice when what they're seeing in front of them, if it looks good or looks bad.
Now, many of my students end up watching my lecture videos
instead of actually coming to class, which
is a common phenomenon in many parts of the world,
as students are increasingly using the flexibility of being able to pause
lectures, slow down, speed them up.
So this is what CS50 looks like.
And this is what Foundations of Algorithms
looked like my very first year when I joined
and was using the built-in projector system and lecture capture in the room.
And depending on what kind of institution you're looking at,
you may have this, or you may have nothing whatsoever.
So you can't really see what's going on there.
So I didn't think my students would rather learn from that.
But within about a year, I was able to get something that
looked much clearer and much better.
And depending on how much you like playing with cameras and toys,
you can go on a wide spectrum from I'm going to have a nice webcam
and I'm going to shine a light appropriately on my face
so that students can see me all the way to a multi-camera production.
And so because I'm one of those more techie people,
I thought, you know, let's figure out how
we can do this slightly more sophisticated recording
setup on a much smaller budget.
Now, again, what I'm about to talk about I did with not a tiny budget.
I spent a little bit of money and built a medium budget.
But you can do this on your own sliding scale for what's appropriate for you.
You don't need to go all the way to CS50 the way Harvard does it.
You don't even need to go all the way to what I do it.
Again, the message is, what are the small changes you can
make that will have the big outcome?
So our goal here in this component was to make
the lectures memorable, accessible, and high quality
by using video and audio production techniques to take another step up
from what they were getting before.
And this is something that CS50 has thought very heavily and very carefully
about, including in this one paper that goes beyond just the fancy cameras
to talk about elements of theatricality that get brought into the classroom.
How to take techniques from drama, production design, staging,
that sort of thing, to make students feel
like they're more immersed in what's going on and not just sitting
watching someone stand at a lectern, as you might be doing for the next half
hour, giving a bit of a talk.
So to actually bring them into the world of the lecture.
So one way that CS50 does this, and that we do this fairly affordably,
is to use props.
And these props don't have to be super fancy stuff made
by a special prop designer.
They can be small.
You can see in this photo, potentially, one of the students
is holding up a number two.
And I'm teaching sorting using house numbers
that I bought from a local hardware store for about $5 each.
Now, while, of course, you could go bigger and better, even
this small change, students already felt like they were more
immersed than just drawing the numbers on the whiteboard.
On the right here, you can see the big blue brain.
And this was part of an art exhibit that was being thrown away
after the art exhibit was being done.
And I thought, you know, I could bring this costume into my classroom.
And our students get excited whenever we bring out the big blue brain
because it means that a student volunteer is
going to be selected to come up to the front of the classroom
and do some exciting demonstration.
So the big blue brain itself doesn't have any inherent educational value.
But by incorporating some elements of design and creativity
into the classroom, you can get students excited about the content itself.
And eventually, these theatrical elements
become representative of substantive course elements.
So by creating that link between the fun stuff and the substantive stuff,
you can begin to create links in the students' minds
that actually enable them to learn the material better
and to get excited about the material.
So this is what it actually looks like up close from my little control panel.
Now, I don't have as good a way of hiding what's going on,
but this actually makes it handy to explain how I'm running my class.
So you can see the same kind of thing that's
happening in CS50, just with the cables hidden away a little better.
So essentially, what's going on-- this is a very schematic stick figure
diagram of it-- is I have two laptops, one of which is controlling the code
and one of which is controlling the slides.
And then sometimes I'll plug an iPad in.
And then I have a cheap little device-- you can get these for as little as a few
hundred dollars--
that enables me to switch what's on the screen at any given time.
And just by pressing a button, then I can quickly
switch between the code and the presentation material.
And this ends up actually saving a lot of class time.
I calculated it saves me probably six minutes
per lecture of futzing with things and switching between things.
And this is an extraordinary amount in a lecture that's only 55 minutes long.
So by investing a little bit in technology, not only does it make
the presentation look cleaner because students
see these fancy transitions between the slides and the code,
but it actually saves time and allows me to focus on the teaching
and a little less on the technology, even while requiring a little more
thought to begin with.
But you don't necessarily have to do that.
But if you do want to, it does cost a little bit at the end of the day.
So this was my personal investment into the class
with a mix of camera equipment.
Now, again, what I really, really want to emphasize here is you don't have
to spend the $14,000.
You don't have to spend $140,000.
You could do this with $140.
And just the types of things that you buy, depending on your situation,
will be a little different.
And so just like me, you might soon, too,
be lugging a suitcase full of items from your office
to where you actually end up teaching, whether that's props,
whether that's cameras.
Bringing extra stuff into the classroom can help enhance the experience.
As well after the lecture, one thing that I
do that I find very effective as an educator
is actually going back and watching a recording of what I've done.
Now, I do spend a lot of time editing what I've done
and finding the bits that worked.
And one of the surprising things is I find
sometimes saying less actually results in students learning more.
So sometimes I'll have an explanation that missed the mark.
And if I cut that out and make a note to myself the next time-- next time,
don't give this explanation-- sometimes that's just
as good as adding an explanation.
So while the editing process is useful for producing
this very polished finished product, just like happens at CS50,
the more important part of that process is not actually the video editing.
It's watching back myself and then making changes based on what I see.
And this is, again, as someone says in the comments, less is more.
So figuring out what are the important things that
are actually getting things across.
I also see a question.
What software do you use for video editing?
I use DaVinci Resolve, and I believe that's also what the team here uses.
So there also are time costs that I pay to make this happen.
And this is some of that extra 5% investment,
might be a little more than 5%.
Oops, there we go.
So I do invest substantial extra time in my classroom
getting set up for it every time.
Right now, this looks like an hour of additional set up and tear-down time.
There's me setting up all the cables myself and my student production
associate.
And this results in about an extra $3,500 dollars of staffing costs,
which might be relevant if you're at a university budget where you have
to think very carefully about your TA time.
And on top of that, about two hours of my time
for every hour of class time in doing post-production,
so doing this video editing.
Now, one thing that I should note is while this is my current version, when
I started doing this, I did not have an extra hour of setup and tear-down time.
I had the traditional five minutes in between classes.
The hour that I have now is just so I can do it in a relaxed and less frazzled
way.
And I've already lost more hair than I would like doing it the five-minute way.
So giving myself an hour is now my preferred method.
And then, of course, there's the fun aspect of creating catchy--
these are YouTube thumbnails that I've used for some of my slides.
But more of the point here is creating a fun theme
for every class that kind of hooks students in.
And there are different ways of doing this,
whether this is a demo at the start of the class or a fun thumbnail
that you post to students to get them hyped before the class begins.
This is another method of drawing students in.
[BEGIN VIDEO PLAYBACK]
[MUSIC PLAYING]
- This is the magical star of dereferencing.
The power of the magical star of dereferencing
is to open Bruce's mailbox.
So open Bruce's mailbox, take out what's inside.
[MUSIC PLAYING]
OK, what do you see there?
- 0x50.
- The other thing that happens when you use the star of dereferencing
is you now go to the location on the paper.
[MUSIC PLAYING]
So 0x50 matches 0x50.
And we read out what you find there.
So what do you find?
[MUSIC PLAYING]
[END PLAYBACK]
SHAANAN COHNEY: So again, this is just another example of the type of content
that you can actually put together pretty
quickly if you are recording your lectures
and then get students very excited.
So I'd say that was about another 20 minutes of effort after the fact.
And students then started sharing that around with friends
and encouraging more people to come to the class, which is, again,
beneficial to the environment overall.
So while I've spoken a fair bit about production and technology, I've said,
you know what if you buy this other camera, what if you film your lectures.
These are all cool, and they're definitely added bonuses.
But I would say that they're definitely not enough on their own.
You can have the world's most beautiful looking lectures,
but if you don't have the other ingredients,
you haven't actually done anything other than create
a very slick looking YouTube video.
So here is an example of actually taking some of these elements
and combining them for pedagogical value.
Now, CS50 has done a number of similar exercises in the past.
I'm going to let it run for one minute so you can see what kinds of things
you can do in the classroom with relatively simple props that actually
use the props as a way to hook students into the material you're teaching,
and really deliver pedagogical value.
[BEGIN VIDEO PLAYBACK]
- So first card and then the second card.
The first card will close, then the third card.
And then the--
- Second card closes.
- Fourth card.
And then they will switch.
And everything--
- Over here.
- OK, then the first card.
- OK, you guys are getting the hang of it.
Let's speed it up slightly.
We're almost through.
- Third card, fourth card, fifth card, sixth card.
That's right.
- Yep, looks good to me so far.
- Card, and eighth card.
[LAUGHTER]
All right.
OK.
First, second, third.
They switch second and third.
And then first and second switch.
Yeah, that's sorted.
I think we need to go through and verify.
So one, two, three, four, five, six, seven.
- OK, everyone turn around.
- And that's right.
- Can we have a big round of applause for Disan?
[APPLAUSE]
[END PLAYBACK]
SHAANAN COHNEY: So while that is a simple sorting exercise,
bringing in these giant human-sized playing cards
suddenly made the experience a lot more tangible, a lot more exciting,
and a lot more memorable through this technique of theatricality
that CS50 uses that we could bring into our own classroom,
despite not actually being CS50 itself.
And you can do this kind of thing, whether it's with sheets of paper,
like David talked about yesterday, with giant playing
cards, house numbers that you buy from the hardware store,
or whatever it is you use.
The techniques of theatricality can be brought into your own classroom
with only a little bit of extra creativity
and a little bit of extra effort.
So one thing that I do that's different from CS50
is I do use a lot of cold calling in my classroom,
trying to remember students' names.
And I find that for my classroom, and at least with the number of students
that actually show up, this is a very helpful approach
for actually getting student involvement and bringing students
to the front of the class.
So this is one of my ingredients that's very highly debated
and educators have very different approaches
to this because students will sometimes have a negative reaction.
But I've found it's valuable in my classroom at least.
So how do you actually do all of this?
How do you learn how to do all of this?
How did I learn how to do all of this?
Well, there's one surprisingly simple answer to it.
I spent a lot of time on YouTube University.
So this is whether it's looking at CS50's material
or looking at other educators or looking at content creators
on the internet who are talking about how to set up a camera so you
don't get glare in your glasses.
There are so many free resources out there for teachers now.
And I would encourage you to actually spend some of your time
as professional development going out there and actually looking
at what other people do and then taking the best elements that are going to fit
your classroom and bringing them in.
I've also found that the listserv for teachers, the email listserv,
has been an incredible source of support.
And CS50 staff have been very helpful if you
do start running against challenges with adopting
some of their tools or their pedagogy and have
been very helpful and very responsive.
So that covers the pedagogy and the engaging delivery and active learning
components.
We spoke a little bit about active learning yesterday in Guy's section.
But there's also some of the tooling that CS50 uses.
And this can be adopted and adapted as well.
So CS50 has their website.
And they have, again, much like many of the other things for their class,
made the source code available.
And so I was able to adapt that and produce a similar looking website
for my own class building off their code, which now enables
my students to have the same kind of positive experience accessing
the material beyond the learning management
system built into the university.
Now, again, your mileage may vary depending on your institution's rules,
but I found that adopting some of the code
that CS50 uses to run their infrastructure very, very helpful.
One other thing that's different from what
CS50 has done that my group has done is tried to do some more automated style
checking, where CS50 is traditionally relied on humans, even though the AI
grading is coming more and more into a part of their infrastructure.
So this was a paper that my team presented
at SIGCSE, which is a computer science education conference in the last year.
And we built a tool that aims to provide students helpful error
messages to understand not just how to fix their bad style in code,
but also what might be a pedagogical learning from it.
And so while the comments that I've shown here
are reduced in size, if you were to click on any of these,
you would see an expanded explanation explaining to the student what
this element of style actually means.
So again, this is an indication of where tooling can actually
make you more efficient as a teacher, giving you more time
for the things that matter.
So a 5% investment in technology in this case
allows us to spend more of our TA time on the one-on-one student
interaction and the things that are harder for computers to deliver,
even while AI tools are advancing and actually
allowing us to do more and more of the substantive hands-on stuff
and replacing more and more elements of what TAs would have used to do.
So, again, with this, the feedback that we got from TAs was universally saying,
I now don't have to spend as much time on the menial tasks and the things that
don't need me, and I can spend more time on the things that
are going to make the real changes to the students.
Helping them through whatever their substantial struggles are.
Helping them cover gaps in understanding and sometimes even
being a helpful ear to them when there are
things going on outside the classroom that are impacting their learning.
So another thing that you have just heard
about that I've managed to integrate into my own class
and I want to give a positive report of that
is using CS50's AI tools to help answer questions on our class discussion board.
Now, you might be surprised, you went through a session before,
and you spent a little-- it took about an hour of building code.
And you might not feel super confident that this is something
that you could roll out tomorrow.
But one of my experiences was CS50 gave this talk, a workshop at SIGCSE
earlier in this year as well, and I was very excited.
And within eight hours of hearing the talk,
I was able to roll it out into my own class.
And so this was a very powerful-- again, adopting what CS50 did,
but using slightly different configuration on the back-end
so that I was able to roll it out with a little less effort and a little
less time than they necessarily did.
So this was again, one of these very effective 5% effort,
but very substantial outcome very quickly.
And we found our outcomes were quite similar to CS50 adopting this tool.
This is also true of the programming environment,
kind of CS50 Codespaces, that has been incredibly effective.
So we've experimented with Codespaces.
Ultimately, in our environment, we decided
to go with edstem, which is a tool that CS50 has used in certain contexts.
One thing that I'm actually a little worried about
as an educator being direct is I don't like having too many of my tools
reliant on infrastructure that I don't control.
The reason is, no matter how good a company is,
I'm always a little bit worried that one day that something might
happen to that company, and they won't be able to support me anymore.
So that's one thing that I very much appreciate about CS50's tooling is
that even when they use something, like Github, which is incredibly helpful
and has been a great partner, they also retain
a lot of control over the infrastructure and are able to deploy it
somewhere else should they need to.
So edstem has been very useful for me, but I
am looking towards adopting some of CS50's cloud tools.
So beyond the technology and infrastructure,
another thing that makes CS50 very, very powerful for students
is this sense of community that's generated
through many of their activities.
[BEGIN VIDEO PLAYBACK]
[MUSIC PLAYING]
- Hi, everyone.
Welcome to the University of Melbourne, and welcome to the School
of Computing and Information Systems.
I'm Jud from CISA.
And today, I'll be taking you on a tour to the First-year Learning
Center located in Melbourne Connect.
Every day, there'll be tutoring for Foundations of Computing and Foundations
of Algorithms, and snacks will be provided.
Come along.
[MUSIC PLAYING]
So this is Melbourne Connect, where the First-year Learning Center is located.
So the Student Learning Center is located on level three.
- Car G.
[MUSIC PLAYING]
- It's located just down this corridor here.
[MUSIC PLAYING]
And it's just here.
Welcome to the First-year Learning Center.
[MUSIC PLAYING]
[END PLAYBACK]
[MUSIC PLAYING]
SHAANAN COHNEY: So one thing I wanted to communicate
by showing you that is if you actually look
at the physical space in that video, there's
nothing inherently exciting about it.
It's a room in an office building.
But the idea of marking it out as a unique space for students
and trying to create that sense of community through things
like having snacks there on a regular basis.
Through having our office hours in a consistent location
has over the year, or the year and a half since I've been running it,
created an enormous vibe of connectedness that previously was not
part of our major altogether.
Now, it's been incredibly useful from an educational perspective
because we now have a drop-in center where
students can get one-on-one support.
But a very substantial outcome from it has
been this notion of students feeling more tied to the place
that they're learning, which, again, actually ends
up improving their learning outcomes.
Another way that we work to build community
is by partnering with CS50 on some of the events
that we don't necessarily have the resources to plan ourselves.
One of these that's been a big success for us
has been CS50 Puzzle Day, where the CS50 team puts together a sheet of puzzles
that aren't coding problems.
They're just logic puzzles and games that students compete in teams together
to finish during a set time period.
And having these kind of extracurricular activities
helps to build community, which then plays back into the classroom itself
and makes the internal classroom work a lot better, despite the fact
that these things don't inherently have a connection to the material.
So our outcomes for developing this center over the last year and a half
are pretty incredible.
So we've managed to get eight hours of student clubs coming in
to help out other students, along with the 20 hours of formal support
that I was able to arrange.
And within just one year, we had more than 4,000 individual student visits
to the space and 1,000 one-on-one support sessions.
So these numbers are pretty substantial for our class, which
means that each student was, on average, accessing
this space at least a few times during the course of the year.
So the last thing that I haven't quite solved,
and I would love to say, you know, I have managed to get all the way to CS50,
but the truth is all of us are striving.
Even CS50 is making changes year-on-year.
And one area where I still think there's a lot of work for my particular class
is assessment.
I am constrained a lot by the particular rules of my university.
There is a nine-month long approval process
for any change in assessment structure, which definitely slows down innovation.
However, I have had success in moving from large exams
to more CS50 style weekly assignments and weekly problem sets
that we found really does ensure that students get started a little earlier.
And that is generally considered to be good educational practice.
So we often talk about, in the world of education,
the difference between formative and summative assessment, where
summative assessment gets students to show what they've learned at the end.
Whereas formative assessment is about building their skills earlier on
in semester.
So by reducing the stakes and the amount of value
that students place on the summative assessment
and shifting some of that to the actual learning process,
we found that students are feeling much more satisfied and indeed,
their scores are increasing.
So is this all worth it in the end?
There have been a lot of changes that I've made.
I've put in a lot of time.
Even if it's just 5% of what CS50 is doing, we still have to ask, is it worth
it?
So are we getting this 80% of the outcome on 5% of the resources?
So just like any other teacher, I've got the survey,
end of semester survey things.
So the results did go up very markedly.
But as we know, surveys are not really the best way
of testing whether students learn.
If you make your assignments easier, your assessment,
your end of semester scores go up no matter how good a teacher you were.
So this only tells us so much.
Most of the best outcomes that we find we're getting
are actually outside of the traditional measurement
instruments of surveys and grades.
We've had a lot of very positive student feedback,
including long thank you letters coming back years,
students saying, you know, I took your course once three or four years ago,
and now, I'm watching the lecture videos every year
because I get really excited by the material.
So even that, showing that they're still engaged in the class material years on,
we consider it to be a substantial success.
Another comment that we found particularly heartening
was students feeling that they're now part of a community,
whereas previously they felt like they were individual learners.
We've also had some limited success on our YouTube channel
where we've been publishing these for the world.
So this was an example from last year where we had double the amount of views
than we had students in the class.
So that's showing that some people are picking this up.
And that shows in our YouTube comments as well that are pretty positive.
And it's great to try and share our educational innovations
with the rest of the world, just like CS50 shares theirs.
So a good way to assess whether what we're doing works
is comparing my version of the subject against a different version
that is taught in the alternate semester by a different professor.
They have different teaching staff.
They pre-record their lectures and do flipped classroom style and use a little
more--
they lean more heavily on PowerPoint.
But it's much cheaper for them to deliver.
They don't have any of this production setup.
They run the same slides and same videos year-on-year,
but it scores very similarly on the survey.
And so if it's scoring similarly on the survey, of course, I'd be asking myself,
do I think this is actually worthwhile?
Another thing that I might be worried about
is that the better my videos become, the fewer students
I might actually have in my class.
But experimentally, empirically, I haven't really
found that my attendance has changed.
So let's think about student learning.
Maybe this will help answer whether what I'm doing is worthwhile.
So you remember the 2020 curriculum?
It was six weeks of introduction to programming
and then six weeks of basic algorithms.
This is what it looks like for my version of the class in 2023.
So now, we only spend three weeks on the C programming
with talking about algorithms still during that process.
Then I spend six weeks on algorithms and data structures,
covering much of the same material as before.
But that only takes us to week nine.
And we've got 12 weeks in the semester.
So then I can incorporate an additional two weeks of advanced computer science
theory and present that in an exciting way,
such that students get excited not just for the practical and engineering side,
but also get excited for the theoretical components of computer science.
And then we have one week for fun topics, which
might be web development or AI or cybersecurity,
just like CS50 includes extra fun topics throughout the course material
to keep students engaged.
And we've actually found that despite students now
facing some more complicated material and indeed
material that students often turn away from because it might
be a bit more mathematically dense, satisfaction and grades have actually
gone up.
So we only assess nine weeks of content, which
might be why students feel more comfortable with some
of the more mathematical and more theoretical topics
that we provide at the end.
But we've still found that when we have conducted ad hoc assessments later
that aren't part of their grade, that students have actually
retained the material that we're teaching in the later weeks.
So why do we teach this extra material and not
assess it in a summative fashion?
Part of the idea here is that we want to encourage students
to be interested in computer science without the threat of them worrying
about their grades at this particular point in time.
So even by exposing them to material in an engaging way,
I think we are achieving something as educators.
And so that's something worth remembering.
Not everything needs to be contributing to the final test.
Something is just going to be interesting or extension material.
And as educators, it's important not to lose sight of that.
So we're almost at the end here.
But think about the 80% in 5% of resources changes
that you've already made.
And the takeaway from this talk, the biggest one,
is I want you to go home and think about what the changes that you could make
are that would deliver similar larger outcomes with not so much input.
So I'm going to give you-- aside from that biggest one,
I'll give you a few smaller themes that are my takeaways at least.
And the first is that you don't need an entire institution
backing you to make very big changes.
You can do a lot with only limited support for your definition of limited.
Whether you are a single person teaching a small classroom in a town somewhere,
or whether you are at a large university in higher education.
No matter what the level of support is, there is always a lot you can do.
And it's generally more than you'd expect and more
than your colleagues might expect.
It just requires you to be a little more creative.
And the more time you spend putting into your creativity, the more return
you'll get out of it.
So make time to not just be doing the rote stuff of the class,
to be making lecture slides or planning class or working on assignments.
Spend some time explicitly dedicated to trying to come up with new ideas
to try and think of ways to do things a little differently that step
a little more outside the box.
And these are often places that, if you make a change,
will result in a larger outcome that makes students very excited.
And then one other takeaway is that if the thing that you're trying to improve
and the thing that you care about is end of semester survey scores,
this is not going to be the best way of doing it.
And so really, maybe we should think about our goals.
It is not the best way to improve survey scores,
but it does increase the joy in teaching and learning.
And so let's throw away the surveys and think
about what 80% changes we can make with just 5% of resources.
Thank you very much.
And it's a pleasure to be here with all of you.