KEVIN XU: Hey everyone!
Welcome to our introduction to ML seminar.
I'm Kevin Xu, a sophomore at the college,
studying computer science and physics.
ZAD CHIN: I'm Zad, I'm a sophomore at Harvard College,
studying computer science and maths.
KEVIN XU: And we're so excited that you could join us here today.
We know that we're in the middle of final test week,
and so everybody's just a little stressed.
So, hopefully, we can give a very interesting, and hopefully
fun presentation about what to look forward
to as you start implementing your final projects.
So just some logistics first.
We will have some set points where we'll take a few questions from the audience.
So feel free to type them into the chat as you think of them,
and somebody will probably read them to us when we pause for questions.
Yeah, other than that, let's just get straight into it.
Right so, of course, our seminar is about machine learning, or ML.
And, so, the first question, which is not so obvious,
is what is machine learning?
And, so, I'm sure a lot of you have heard about the developments that
have been in this field, a lot about neural networking, and perhaps
reinforcement learning.
And a popular topic, of course, is the game development theory,
where some computers have solved games through these complicated machine
learning algorithms and neural networks.
And, so, we just want to give a quick overview of what
exactly falls under machine learning.
And this is actually a very broad category
that includes a lot of different--
some ideas, and some fields.
And, so, there are three main ones.
There's unsupervised learning, supervised learning,
and reinforcement learning.
And a lot of the neural networks that you think about,
when you think about machine learning, and reinforcement learning,
fall under one of these categories.
But, at the end of the day, ML is about taking a lot of data
and having the computer, or an algorithm, or a program, process
the important parts of that data.
Recognize the patterns in the data, and attempt
to use that data to generalize to a bigger data
set that you might be given.
So Zad now is going to present a little bit more about what exactly
these subfields in ML are.
ZAD CHIN: So the first thing we want to know is about supervised learning,
so what is supervised learning, right?
So, given some labeled data, say for example,
we have a data set of cat and dogs pictures.
So, how can a machine learn to predict the labels that
generalize to unseen data?
So what we do is we put a data set of cats and dogs
pictures, and then put it into a machine learning model.
And let the machine learning model kind of learn itself on how
to differentiate between cats and dogs.
And then we have some testing data set, we maybe have cats and dogs, or maybe
other images, and the machine would kind of like label, oh, this is a cat,
or this is a dog, or this is not a cat or a dog.
So this is kind of like a supervised machine
learning, whereby we actually label the data so the machine can learn from it.
So, the next big idea in machine learning
is also unsupervised learning, where the machine actually recognizes
the pattern itself from the data.
So like, we don't label anything, we don't label it's a dog, or a cat,
we just show all the images to the machine learning,
and the machine will be like, oh this feature kind of like resemble a cat,
so this is a cat.
And this feature kind of resembles a dog, so it's a dog.
So that's kind of like a difference between unsupervised learning
and supervised learning.
And most of the time, unsupervised learning
includes like, clustering for example, in a very high dimensional ICU data.
Like who is most likely to get readmitted,
and what are the features that those people are going to be readmitted
or not?
So that's kind of like a difference between supervised and unsupervised
learning.
And next, Kevin is going to talk more about what is neural network,
and why it important to unsupervised learning.
KEVIN XU: So, as Zad mentioned, a lot of the time you're dealing with data
sets that are hugely multidimensional.
And all that means is, there's a lot of different categories that you could
consider, such as an election, like the population, the demographics, how
likely the other candidates are, et cetera, et cetera.
And you just have all of these categories of data
that you need to somehow compile together and get some reasonable kind
of prediction as a result.
And this is where neural networking shines.
And if you continue researching this field,
and perhaps continue with this kind of topic,
you're probably going to see this kind of picture a lot.
And at first this looks very incomprehensible.
It's just a giant graph with a bunch of nodes and lines.
But this is actually just a very brief picture of what neural networking is.
On the left, you can consider these as input nodes, and on the right
you have output nodes, and in the middle you have hidden nodes.
But this doesn't really tell you anything about what this does.
And, so, you can think about this giant network of graphs,
or however you might imagine it, as just a black box function.
And this black box function takes in your input data,
so the different categories, your multidimensional input data,
and then it spits out the output data that
should be good enough for a prediction.
So, in the most simplified case, let's just
talk about a game, such as tic-tac-toe.
And, so, your input data just might be the state that you're currently in.
So, what squares are filled, whose turn is it, et cetera, et cetera.
And your output data might just be one single number,
the heuristic between zero and one.
And all this number tells you is how good your current board position is.
So obviously, if you can make three in a row in your next turn,
your heuristic should be very good, it should be one.
And if you're going to lose in the next turn it should be zero.
So, obviously, not every state falls under this case.
So, there's a continuous range of numbers between zero and one
that your black box function should be able to retrieve, or accurately
predict from this input data.
And this is a job of everything in the middle, right?
How do these variables relate, what function
or what functions you should apply to these variables in the middle
of processing, et cetera, et cetera.
And, so, the overall goal of neural networking
is to design this middle function, right?
How do you get the computer to find these patterns, and to make--
create this black box function in a reasonable manner.
And this is something that's really hard for humans to do,
because there's so much data.
And, so, we give it to the machine instead.
ZAD CHIN: So, a general outline in a machine learning
project, or machine learning research, for example,
we start with a data collection.
So there are a lot of different ways where we can collect data.
Some significant ways like finding sources of data
is like, scrapping it from websites, API call, or readily available data
sets such as Kaggle, or like in UCI ML repository.
But do take a note that scrapping raw data, for example, from API call,
or even websites, may take a long time to clean up
the data, especially when there is a lot of empty rows and stuff.
So the next step we do is normally data exploration.
And initial data exploration is normally very powerful,
and it gives you great insight on what features to take in,
what features to drop, whether your data is biased.
And understanding the dimension of the data is actually very important.
So the third one is, of course, choosing an ML model.
Choose a simple model to start with, say for example,
what is the type of problem you want to solve, right?
We'll go too deep in that-- in an example
later, but like, some problems I consider,
is it a regression or classification problem?
Or whether to actually use the supervised or unsupervised machine
learning model.
There are some models that you can choose from.
And the fourth one, after you choose a model,
and code it out with all the libraries, you get the result.
You want to test it out.
What is the accuracy score of your model,
what is the precision, what is the record score of your model.
And how can we improve the model by fine tuning the parameters
or even using the more sophisticated models?
Say, for example, you started with logistic regression,
maybe you can move on to neural networks.
So, I think we can start with a bit of questions if you guys have any.
Or not, we can move on to examples that we have.
SPEAKER: OK, there's one question, from Prateek.
"Is there any difference between data science and machine learning?"
ZAD CHIN: So--
KEVIN XU: Yeah, do you want to take this, Zad?
ZAD CHIN: Yeah, sure.
I think data science is more--
I think machine learning is under data science.
You can correct me when I'm wrong.
I think, like, data scientists somehow also use machine learning model
to help them to analyze the data.
So, basically, what it means for data science is that like,
we try to understand the pattern, or what useful information
can a huge amount of data tell us, right?
So, I feel like, machine learning model can be a very good stepping stone
into data science.
And also, there are statistics as well, like pure statistics,
such as chi-square test, which is not really machine learning but also
a part of data science.
KEVIN XU: Right, just echoing what Zad said, ML is a technique.
Or a-- you can always think of it as a methodology
to approach the study of how you can extrapolate data.
And, so, there's a lot that falls under this, that falls on machine learning.
But there are also a lot of things that you can do
that are not machine learning, right?
And, so, I hope that answers your question.
SPEAKER: OK, no other questions, you can continue.
KEVIN XU: Zad will have to [? click. ?]
ZAD CHIN: So you guys can go to this link.
If you want to make a copy of the notebook yourself and follow up when we
actually present it, you can go to this tinyurl.com/ML-notebook-1.
We will jump straight to the notebook that we can show you.
So there are a few contents, if you want this table of contents.
And I'll close it, so we can have a better view of [? it. ?]
So let's get started.
We talked a lot about how we can start with machine learning research.
So, in this particular notebook, we will try
to use the Iris data set, one of the most famous data sets
about machine learning.
So, in a supervised learning, we will try
to use two different models, which is k-nearest neighbor
and logistic regression.
And, the second part, we will jump into reinforcement
learning with a different example.
So, as we talked about last time, the ML development process
includes data collection, train-test split, identify correct ML problems
and ML model to use, evaluate the performance of the algorithm,
and try different models and retrain to retest.
So let's get started.
The first thing you want to do is, of course, import the necessary libraries,
like Pandas, NumPy, plotting library, train-test library.
So you get straight and run this.
So to run a cell in Jupyter Notebook or like, Google Colab,
you just press Shift-Enter.
And, so, the next thing I'll do is to import
the data set that we talked about, which is the Iris data set.
So this DataFrame.head is to print out like-- or how the--
it's to give you an idea on how the DataFrame actually looks like.
So we can see that if we run this, there is sepal length, and then petal length
and petal width, and there's target.
Target is the three different things that we have, which is, I think,
different kinds of iris.
And the next thing we want to do is try to explore
what is the shape of the DataFrame, for example, right?
Now, we know that there are 150 rows and five different columns,
here we can see there's five different columns, and there's 150 rows.
So, first of all, the first thing we want to do
is to actually explore and analyze the data.
That's because it's important to do initial data exploration because it
might be biased, it might be noisy, and it
shows the relationship between different features and target
and better train our ML.
So I put some links here that you can use next time.
But we will start by using-- putting a pie
chart between the different targets.
Like, there's setosa, virginica, and versicolor.
That's a different type of iris.
I can see that each of them, kind of like,
33%, represented very equally and balanced in the labeled data set.
And this is to plot the number of different labels versus--
this is the number of like-- count, and then this is the width.
So you can see that most of the--
say for example, sepal, we can see fall in the 3.0 to 3.5,
and it's kind of evenly, or normal distributed.
And this is more like a graph to see how different kinds of length and width
vary according to species.
And I take some time, and I think this is very useful.
And this is basically a pairwise plotting
between how the features and the target are influenced.
It takes some time to plot, but it will be cool.
So say, for example, look at this.
We can see that like, petal length has a very huge influence on the target.
You can see that for target zero, they have really,
really, significant smaller petal length, as opposed
to target two which actually has a very clear distinct feature.
And moving forward, to actually-- you know like sometimes,
when you look at graph, it's not intuitive?
So a very, very good way to plot the intuition--
to calculate, to see the relationship between different features,
is to plot a heat map.
You can see that when we plot a heat map,
you can see the higher correlation between petal length and target.
This is where we want the label, right, petal length and target,
and petal width and target.
So we pass it into the machine learning model where you want to-- we can also
drop-- we can actually drop, which means delete,
the column of sepal width and length, because it doesn't really affect--
it doesn't really correlate to the target that we actually care about.
So the next thing we want to do is to train-test split.
The importance of train-test split can be learned more about here,
but we will just move on.
So, basically, when we have a data set, we
can't just put everything in your training sample.
We need to split it into maybe 70/30 split, which is 70% training, then
30% testing, or 80/20, 80% testing and--
I'm sorry, 80% training and 20% testing.
Depending on the size of your data set, some people
go with 70/30, if you have a huge data set you can actually do 80/20 split.
So, in here, because we don't have a huge data set,
you only have like 1,500 rows, so I decided
to go with 70/30 split, which we can see that the test size is 0.3.
And we have, after splitting, we get 105 rows in the training data set,
and 45 rows in the testing data set.
And we can also check whether we actually trained it correctly,
and this is a very important step whereby we actually split.
So, now, we have five different columns, right?
We have sepal length, sepal width, petal length, and petal width.
And we have target, and this is what we want to test it out.
So we need to split the training data set into both x and y, where we have--
these are call features, and then this is the target that we want.
And these are all features that are in the testing data set
and this is the target.
So we split that, all of this can be fine--
this is actually under Pandas, if this notation seems really, really weird
to you.
It is very well documented in the Panda DataFrame.
And this is still head, and we see how it looks like in this
or-- after I split it, you can see that train x now has four things.
Four different features that we want to train it on the head, which is this
is the attribute of it, which correspond to the ID features.
So, next, what you want to do is identify correct problem and ML model.
So in this problem it's more like a classification problem.
So, for example, if you are provided with several inputs and variables,
a classification model will actually try to predict
the class of the kind of data.
Say, for example, here is a classification problem,
because we are given the features sepal length, sepal width, petal length
and petal width, and we want to find what target, which
is what iris species it is.
So that's a classification problem.
As opposed to a regression problem, where we are given certain features.
You want to find a petal length or petal width, that's a regression model.
You can read more about it here, I'm not going through it for time's sake.
And before we move on to test out that model,
I want to talk about evaluation metrics so we can keep running and see it.
So there are a few eval-- before we dive into evaluation metrics,
we need to understand what is a true false--
true positive, true negative, and false positive, and false negative.
So a true positive is when outcome correctly predicts the positive class.
Say, for example, you have a one, and then the model actually correctly
predicts the one class.
And, similarly, a true negative is an outcome where the model actually
correctly predicts the negative class.
Let me give you an example.
So, basically, say we have a cat data set and a not cat data set.
So a true positive is whereby a model correctly predicts that it is a cat.
A true negative is when the model actually correctly predicts
that it is not a cat.
So that's the difference between true positive and true negative.
A false positive is an outcome where the model incorrectly
predicts the positive class.
Say, for example, I have picture of a cat,
but then my model predicts it as not a cat [? view. ?]
That's a false positive.
And a false negative is an outcome where the model incorrectly
predicts a negative class.
That's when I give--
I feed in a non-cat picture, but the model says, this is a cat.
So that's a false negative.
So in terms of a different kind of metrics
that we use in judging the performance of an ML metrics,
we have accuracy, recall, precision.
And we also have other called the ROC which is receiver
operating characteristic curve.
And AUC, which most often used as opposed
to ROC, which is area under the curve.
So accuracy bias' meaning is the fraction of predictions that got right.
Say, let's take the example of the cat and non-cat data set, right?
So the fraction of--
out of the 150 prediction, for example, if my model
predicts 96% of the cat pictures correctly, that's an accuracy.
And, also, you have recall, which is what proportion of actual positive
are defined correctly.
So mathematically, it's defined as true positive or true positive plus
false negative.
And precision is a true positive or true positive plus false positive, sorry.
So, it's like, what proportion of positive identification
were actually correct.
And I thought it might be a bit overwhelming
to hear a lot like, true positive, true negative, it's like,
oh my god, this is so much.
I included a lot of different resources that you can actually go and read it.
I think they are pretty good resources whereby you can learn more, and get
accustomed to all of these terms.
And, so, let's first dive in to the first machine learning--
the supervised model that we're going to talk about,
which is k-nearest neighbor.
So the k, in k-nearest neighbor, is the nearest neighbor
we wish to take the vote from.
Let's do an example, right?
So this is a dot I want to label, whether it's a blue or a red dot.
In the radius of three, which is this-- what's it called--
solid line circle, you can see that we have
two red triangles but one blue square.
So, in this case, this dot will be classified by the KNN label
as the red triangle.
And what about if it's larger, right?
So what about k value of five?
So it would be the dotted line circle.
So you can see that in this dotted line circle,
we can see that if we classify this green dot, we have three blue squares,
but we have two red triangles.
So this blue dot, based on this dotted line circle,
will be classified as a blue square.
So we can see that actually choosing a k value in the KNN algorithm
is very important.
And that's where all the time consuming parts come in,
because we want to choose the correct model that doesn't overfit or underfit
the algorithm.
So some of the ways I can do it is plot a graph of accuracy versus k value,
or graph of error rate versus k value.
But, most of the time, we just test with a random one and we move on from there.
And there's also a different metric system that we can use.
If we learn maths we also learn about the Euclidean distance,
whereby it's like, draw a triangle, and we have the Euclidean distance.
Or we have the Manhattan distance, whereby
it's like the longest distance from a point to a point.
So the KNN resources that I found really, really helpful in explaining
will be here as well.
So, without further ado, let's get started with model testing.
So I use a SKlearn model to--
I paste in my k-neighbor, which is three, and I paste a KNN classifier,
and I train it.
And after that I used-- this is the prediction-- after I train it,
I just used the model to predict my testing data set,
to see how accurate it is on the data set itself.
So, after I run it, you can see that the accuracy score was around 93%.
That means that of all the positive--
of all of the predictions that all the ML model made, 93% of them are correct.
And what about the precision score?
We got 0.9476, around 95%.
That means that out of the total positive observation,
95% of the prediction is actually correct.
That's pretty high.
And the recall of the KNN is around 93%.
I'm so sorry, I will change this later.
But, maybe, let's test around with different kind of neighbor scores.
[? Let's do ?] about 10, which is basically overfeeding.
We can see that it actually affects the--
I think, just now, we have 93% accuracy.
But now, we can see that if we feed it with a amount of 10 neighbors, which
is increasing the radius of the circle, you can see that the--
both the precision, and accuracy, and recall score
actually increased from around 93% to 95%.
So that's why constantly testing and evaluating the model
is actually very important.
So, let's move on to another one, which is logistic regression.
Logistic regression is mainly based on this graph,
called the MLE, maximum likelihood estimation.
Giving an example would be, say, for example, I was given a bunch of data
and I was trying to predict whether someone is COVID positive
or COVID negative.
So, what we will do is, we will plot a graph like this, a sigmoid curve.
And then, say the data was here, right, and I'll be like,
oh, because this data was around more than 90, more than half of this curve,
I would classify it as one.
But if it's here, we'll classify it as COVID negative.
So that's how this model generally works,
is that depending on where the data points are,
we classify whether it's positive or negative based on where it is.
So, in this case, you also try to run a logistic regression
model on the Iris data set itself.
And I also include a few resources where you can
understand logistic regression more.
So if we run the logistic regressions I think it's actually--
[INAUDIBLE]
So, if you run the logistic regression, you
can see that the accuracy score is actually less than the one
that we had before.
We have 95% of accuracy with KNN model, but we only have around 93%
with logistic regression.
And that's how we actually, so--
alternatively, you can use other approach or ML
model that we suggested here--
sorry to keep jumping--
which is decision tree, support vector machine, or neural network.
I remember when you run it across support vector machine,
it has really good accuracy and prediction,
while neural network is also a pretty good way to try to classify it.
So that's all for me you can run on to--
KEVIN XU: Yeah, so, great, Zad, a wonderful job
of highlighting how you can use a regression,
or you can use ML to take a data set and attempt to predict future elements
that may be part of this data set, right?
And, so, this is one of the core pieces of ML.
And, don't be worried if you couldn't follow all of that.
Or, you don't really understand how the syntax for the code works.
It's learning new libraries, and machine learning
is very heavily dependent on previously written libraries.
It's a lot of work to develop your own algorithm to-- for the machine learning
take place.
And, so, a lot of the case it's the--
a lot of the time it's the case of seeing how your data looks,
and then trying to choose the best model.
Which is a sequence of algorithms that make this black box function,
as we talked about earlier, to actually get accurate predictions.
So, yeah, feel free to ask questions-- or pose questions about this.
But I will be going on to kind of the flip side of machine learning.
Instead of using previous data to predict elements of the same data set,
we're going to start talking about how can we get the machine to calculate
or like-- we're going to start talking about games, and reinforcement
learning.
And, so, one of the most popular uses for reinforcement learning
is, of course, trying to solve games, right?
So you have a game, and you want to build an artificial intelligence,
or an AI, that best wins your game, or that
always plays the best move possible.
And this is actually loaded with theory about how data sets work, how the game
itself works, and there's a lot of math and logic behind it.
But, at the end of the day, the idea is, given what you know about the game,
can you get your computer to train on the game such
that it always has a pretty good idea of what the next best move is?
And, so, I've built just a really silly game here.
We don't have to worry too much about the structure of the game itself,
but in general, essentially, when you start the game
you're presented with two doors.
You choose one of the doors, and then you're
presented with another two doors.
And you choose one of the doors again, and behind that door,
there's a randomly generated value between zero
and a certain number that corresponds with the door.
So if you think about this kind of tree-like structure,
you have two, and then two more, so there's four total doors at the end.
And each one of them has a number assigned to them.
Perhaps like nine-- or in this case, I think, three, nine, one, and 20.
All right, so obviously, the door that is associated with 20
is going to, on average, give you better points, or reward,
or whatever this point system works out as, right?
Than the door that has a one associated with it.
And, so, the question is, can you get the computer
to realize which door is the best, simply
by playing the game a bunch of times?
And this is the concept of reinforcement learning.
You just play it a bunch of times.
And for things that turn out well, as in you've got a lot of points,
and well, yeah, you've got a lot of points,
then the computer should be more likely to choose that option in the future.
And for doors that you didn't get a lot of points,
you could be less likely to choose that door in the future.
So we don't have to worry too much about the overall structure of this code,
but right now, I have it set up such that the computer just simply
plays random moves every time.
So it goes through one of the two initial doors, with equal probability,
and then it chooses one of the next two doors with equal probability.
So you expect that the expected value is going
to be pretty low, because these aren't in general pretty--
or, yeah.
So it won't be the highest that it could possibly be, right?
So if we run this code, we see we got an expected value about 4.125.
So, on average, the computer is scoring four points, right?
And, so, this is definitely not the best we can do, right?
This is a totally random--
the computer is playing totally randomly,
like this is completely stupid for the computer to do.
So we want to attempt to get it to figure out which doors are better.
And this actually doesn't have to--
this-- to implement reinforcement learning, a lot of the time neural
networking and other aspects of machine learning
are incorporated with reinforcement learning.
So you take in the data, and you pass it through another regression,
or prediction, and that also helps you find out which move is best.
But, in this case, because the game is so simple,
you can simply just hard code the training in.
And, so, if we take a look here, and this will be a little brief,
so if you want to ask about the logic, go ahead in the chat,
but I'll try not to waste too much time trying to explain
what every line of code does.
So for every move you take, you consider the state that the move resides in.
So, the current state of the board, so, which doors you have in front of you,
and which possible doors you can go through.
So, if you start at the beginning, you have the first two doors, right?
So, your state is just at the beginning, and you have two possible choices.
And, so, if you label every state in the system with a heuristic value that is--
that basically tells you the goodness of that state,
like how desirable do you are-- how desirable is it to be in that state
if the goal is to accumulate points.
Then, what you can do is, have the computer simply
go through this, the states, that have the highest goodness value, right?
So how do we actually calculate this goodness value?
Well, we just play the game a bunch of times.
So for every time you play the game, you're keeping an internal track,
or the computer is keeping an internal track, of what the current state
heuristic value is.
And then, it makes a move based on what it thinks
is the best move in this case.
So, here, once we make the move, we then find out what the next state is,
and how good the next state is as a result of the move.
So using this accumulation, we can kind of backtrack and see
how good was the current state that we were in before we made the move,
and how good of a move did we make?
And, so, of course, by incrementing these, right, you
increase the probability to make good moves and you increase-- you decrease,
relatively, the probability to make bad moves.
And, so, if we throw this training in that calculates the heuristic model,
and we recompile, we see we got an expected value of about 10,
which is about twice as big as we had previously.
So the computer has gotten way better at this game.
And if you actually take a look into the data structure, which
I won't at the moment, because it looks really complicated,
and well, it's just a big dictionary, and I don't think
it will help you understand what is going on here.
If you actually take a look, you'll see that the end result
of the training, so the training, we played to the game 10,000 times.
And we evaluated the goodness of every move for those 10,000 games.
And you'll find that after this training,
the probability of choosing the door that leads to this 20 door,
sorry I went a little far, so that leads to this 20 door at the end
is almost one.
It's like, 0.9997 or something like that.
And, so, the computer has basically figured out,
without us telling the computer anything except for the game, right, the game
state, and what the results are, how to beat this game.
And, so, this is the goal of reinforcement learning.
And I think this is a very interesting thing, that has--
there's a lot of application in chess, and go,
and this is the basic core idea of how people have solved these games.
You want to make good moves, is what it boils down to.
Which sounds very simple, but in execution can
be more complicated than it seems.
Let's go--
SPEAKER: Kevin, and Zad, can we take a couple of questions now?
Oh perfect timing, yay.
KEVIN XU: --that's what we're planning on doing right now.
SPEAKER: There we go, all right.
There have been several since the last little break there.
OK, from Angela, "Was Person of Interest a realistic example
of machine learning?"
KEVIN XU: Person of Interest?
ZAD CHIN: I'm sorry, can you repeat the question?
SPEAKER: That's the question, it was probably about the earlier--
maybe Angela, you can write back in the chat?
I'm going to just keep going on and continue answering,
but if you want to clarify that in the chat.
From Maria, "Do you ever feel like machine learning sometimes
have very serious sequences, like in elections for example?"
ZAD CHIN: Yes, we do feel like, in terms of machine learning,
the impact of machine learning model, especially,
I mean it can be both bad and good.
That's why, at the last slide of our slides,
we also talk about how machine learning actually
generates deepfake, or like, privacy intrusion, the alignment problem.
So we actually include some machine learning ethics
that we hope to actually share with you as well.
But generally, in terms of machine learning in an election,
I do think that that's true.
Because a lot of the time, advertisement model in like Facebook, or Google,
they use a lot of machine learning model to predict the kind of person
that you are, and recommend--
recommender system is actually a part of machine learning.
It's a very huge research topic in machine learning.
And, so, if you want to know more it will be recommender system at Google
or Facebook, I would--
SPEAKER: OK, and then we have another question about cybersecurity.
So is that--
I think we can leave that for later, I think you have some slides on that.
Is that correct, Zad?
Yeah, OK.
ZAD CHIN: Yeah, we do.
SPEAKER: All right, so, let me just--
this is from James, "How does a computer or machine recognize objects
by itself in unsupervised learning?
ZAD CHIN: It depends.
So for example, if you say like, a data point, for example, currently
I'm doing research on the ICU data set.
We have a lot of features, and then, so what we do is we paste in the features,
and then it will be represented in like, let me give you a simple example.
Say, for example, we recommend-- we have x and y feature.
And we recommend it on--
we just plot, like we just put the points in.
And then how the machine learning knows is that they try to cluster the points.
One of the very good ways to actually know, like in unsupervised learning,
is clustering.
So basically, say, for example, I took a point, right?
Like what is the neighbor of the point, and how it should relate,
how strong it actually relates.
So, maybe like, for example, the points are very separated,
or the points are like a block, or the points are not related at all.
So, basically, one of the very good ways of unsupervised learning
is also clustering, that's like for discrete data set.
If you say for image data set, for example, most
of the time we use something called a convolutional neural network, which
is CNN, which is basically passed through a lot of [? features. ?]
And I think in CS50, we also have this pset
where we talk about edges detection.
I think it's under the--
it's one of the psets.
And that's an edge detection, whereby we draw out the edge itself.
And that's also a part of the CNN, whereby
we try to pass through different filters and networks.
And then, we draw the edge to recognize what is the image itself.
So if you are interested in knowing about how machine learning actually
understands images, I would recommend CNN.
If you about-- if you like normal, unsupervised learning,
there are a few where we can say here's like, clustering autoencoder,
which is a part of neural network.
SPEAKER: OK, why don't we take two more, and then we'll continue again.
So, Amir asks, "Which step is preferred first?
Data analysis or machine learning?"
ZAD CHIN: I would-- yeah, go ahead.
KEVIN XU: So, this is very context-dependent at times.
But in general, you don't want to take some data set that you just gathered
and immediately throw it into machine learning.
Because, when you take data, in the real world, as opposed to generate data,
there's a lot of noise.
There's a lot of inconsistencies, there is
a lot of things that can go wrong, just when you take normal data.
And, so, if you just throw something random into an ML program,
it will not necessarily get you the results you want.
And most of the time it won't, because there's so much noise in the data
that it's really difficult to identify the patterns.
And, so, generally, when you're designing this ML--
type of sequence of steps, you want to at least screen your data before you
throw it into any type of program.
That way, you can catch early on if things are going to go wrong, right?
And like, in the worst case, you just might
have to retake the entire data set because it's not valuable, right?
And, so, it's kind of like a screening process, at least the initial data
analysis, before you can actually try and be productive with that data set.
ZAD CHIN: Adding it on, I feel like it's very important
to do data analysis first.
This is because say, for example, you get a data of like COVID positive
and COVID negative patients.
And actually it is very dangerous.
So, for example, if your data set is very
biased toward COVID negative person, and you just
pass it to a logistic regression model, and the model would
be like, oh, since 98% of the people are actually COVID negative,
right, then I can just predict, oh nine--
out of the test example I gave, I will just predict everybody as negative.
So I still get a 98% accuracy, which is actually very, very dangerous.
And it's all the way--
why you want to know how many labels that we have,
before we actually pass it to the machine learning model.
Because these kind of biases can happen.
Machines can be just like, oh, because 98% of people are actually negative,
so I can just predict everyone is negative.
And I get a 98% accuracy, right?
So it's very, very important to do the data analysis
before you actually train the data, especially
on this kind of highly biased data itself.
SPEAKER: OK, and the last question, from Victor, "Do all machine
learning models use neutral--
neural networks?"
ZAD CHIN: No.
KEVIN XU: Yeah, no.
So neural networking are--
convolutional neural networking, these two things,
or same, thing but specialized, are a specific subset
of the machine learning that you can do.
And, so, I think you probably asked this before I showed
the reinforcement learning example.
But you don't have to use neural networks at all when you're
trying to get the machine to learn.
It is very-- neural networks are very powerful, because you're
able to take a lot of data and have the computer generate
the relationships between the data.
But it's not always necessary to use a neural network when
you're trying to learn from a data set, or have the computer learn from a data
set, as you saw in reinforcement learning, right?
You can simply have the computer attempt to make its own conclusions based
on what state, or what type of data you give it, and what it wants to achieve.
ZAD CHIN: Right.
KEVIN XU: So, yeah, we'll take some more questions at the end.
So yeah, feel free to continue posting them in the chat.
But, for now, since we've probably hit you with a lot, and things we can do,
and things we can't do, and possible things,
we want to give some tips on what is reasonable to consider
if you're actually attempting this-- to implement this in a CS50 final project.
And, so, Zad has made this great graph here
that goes kind of in difficulty level from the left to right.
And, so, at the very left you have supervised learning
and unsupervised learning, which is--
requires some effort, but not huge amounts of dedication.
Although, this is always context-dependent as well.
And then, reinforcement learning will probably take more time, simply
because you not only have to provide the data,
but you often have to build an infrastructure that
can interpret the data.
So, in the case of the game, right, you have to build something that actually--
you have to actually build the game into Python.
Where the game has to take in an input state somehow,
and it has to return to you the new state and the results.
And, so, there's all this extra infrastructure
that you need before you can actually run any ML,
and sometimes this takes longer than running the actual ML.
I actually spent longer trying to get this infrastructure
to work out than actually implementing the reinforcement learning.
So this is like very--
something to be cautious of, if you're interested in doing something
like solving a game.
But, of course, it is certainly doable if you
are willing to put in the extra time to do it.
And then, with convolutional neural networks, and deep learning,
and some of the higher stuff, we caution against it
unless you are very familiar with this kind of construct.
And it requires some-- quite a bit of in depth knowledge.
So you don't really have to worry about that.
And, so, quickly, I just want to mention how you can actually
implement these things.
So we used Google Colab which runs Jupyter Notebook which
is a Python interpreter that goes line by line.
So the nice thing about Google Colab is that, well, there's two nice things.
One, is that it interprets the line by line.
So you can change a line without having to rerun
the script from the top down, which is great
if your things take forever to run, as is often the case in machine learning.
And the other nice thing about Google Colab is that it's cloud computed.
So there's a GPU on the server end that does all this,
and then returns it to you over the web.
And, so you won't try and break your machine
trying to process like 10,000 images.
But, on the other hand, if you feel comfortable running it on your device,
it's definitely certainly doable.
I ran the reinforcement learning code just fine
on just a terminal on my device.
And that-- those are the kind of things that don't really
take too much processing.
So, yeah, just keep that in mind when you are thinking
about how to actually implement this.
ZAD CHIN: So, next, I will talk about the useful Python library
that we almost--
not-- normally use for machine learning.
The first one we will go into is data.
How do we get data, right?
So, in terms of mining data from online, we
can go for BeautifulSoup, which is kind of like a scrap library
that someone wrote about.
It's all linked, so you guys can press that,
and the slides are available on the CS50 website.
And, also, Scrapy, which is a very good scraping library.
And in some of like, built in data sources that are very well documented,
I would recommend Kaggle, it's a Google platform
with a lot of machine learning data.
And UCI, University of California Irvine machine learning repo--
also a lot of data sets available.
And you can also scrap from websites or API calls,
with BeautifulSoup or Scrapy, or even your own API call.
And, for data pre-processing, I'm a very huge fan of Pandas.
So I highly, highly recommend using Pandas,
like changing your CSV to Pandas, and just work from there.
It's actually much more efficient and useful.
And there's also NumPy and SciPy, which is also like things that you normally
use in terms of calculating mean, median,
and a lot of very useful functions to analyze
the data or pre-process the data.
So the next thing is Python library, which is like visualization.
The most simple basic visualization library that is available online is
Matplotlib, very useful, super well documented,
a lot of examples online that you can use.
Seaborn is a better kind of visualization,
where you can choose your own color map, or better kind of visualization.
The best visualization that I can actually think of, maybe not the best,
but like--
Plotly is a very highly visual--
very engaging, highly visual, very nice visualization libraries
that Python have.
So, in terms of ML models, there are a lot of built in libraries.
You can also do your own ML model from scratch,
from Python, which actually increases your understanding on the model itself.
But if you want to save time, if you just want to use the model itself,
there are a lot of built in libraries.
Say, for example, SKlearn is a very good library for beginners,
they are various classification, regression, and clustering algorithm
that we can just use.
And there is also TensorFlow, I think a Google kind of like--
a Google deep neural network, Python library.
It goes for various tasks, it's on training and inference,
mostly on deep neural network.
And they have pretty good documentation online and on YouTube and really
good tutorials.
We also have PyTorch, where it also runs on neural network computer
vision and NLP.
And we have Keras, I think Keras is by Facebook.
And it's primarily for developing and evaluating deep learning
models as well.
Those are really, really useful libraries that you--
and are really well documented, there are a lot of tutorials
online that we actually recommend, and it also
linked so you guys can actually have a look too.
In terms of ML resources and support from CS50 and beyond,
we actually recommend you to go on Ed if you have any bugs
that you don't know how to fix.
The team-- the CS50 team is really, really happy to help you on Ed.
And you also have CS50 Intro to AI classes
online too, so you guys can have a look at it.
They also a lot of Python libraries documentation online, where I highly,
highly recommend you to look over the examples
before you actually start coding.
Kaggle is a place where all the data scientists go.
They are data sets, they are also example codes that you can try out,
and also competitions that you can join.
Very good place to learn about data science, and more
about machine learning.
And there are a few blogs that we found very useful as tools.
Data science blog is by Medium, and Machine Learning Mastery blog
is a free blog that's really, really helpful.
Of course, we have our favorite, Stack Overflow and GitHub, really,
really good ML resources and support there as well.
So let's talk about the ML ethics that we actually
see, like a lot of questions about what is the best.
Machine learning can do so much good stuff, right?
Like it is used in health care, education, anywhere
that you can think of.
But, at the same time, it's had its own like, danger itself.
So, for example, one of the things that we see the most is deepfake.
And I think Malan actually did a deepfake example video in last year's
CS50, which is actually very exciting.
I highly, highly recommend you guys to watch,
I actually linked it here so you guys can watch it later.
And another thing about machine learning is actually it's a black box.
So it's like, interpretability in machine learning
is a huge topic that a lot of machine learning practitioners
actually talking about.
So one of them is Professor Finale, also a professor at Harvard.
He-- she actually does a lot of stuff about interpretability in AI
and in healthcare as well.
So I highly recommend you to watch the TED Talk with her, if you want to.
And the other thing about machine learning
is AI biases, fairness, we heard a lot about it.
So there is this specific course by MIT that talks
about AI biases and fairness, really well documented video,
highly recommend.
And like, how you guys say we have a lot of fairness, transparency,
privacy issues, that are related to machine learning.
And there's this book that is really good that I wrote here as well.
It's called The Alignment Problem, it's by a professor in UCB.
It's about how we can align machine learning with our human values.
All of this stuff is linked, so you have more--
if you like to know more about machine learning ethics
and how it actually can be dangerous.
But I don't say I don't support it, I was like,
machine learning is very helpful, but we need to be mindful
that it can be perilous as well.
So, that's all from us, and we will take some questions.
And, yeah.
SPEAKER: OK, wonderful.
So, let's go to Doris, "Is there a need for bigger capacity of laptop
for machine learning?
I'm using an old Mac."
KEVIN XU: This is actually a great question.
This is truly dependent on the data sets that you're working with.
A lot of the times, at least in the universities,
right, when they're dealing with huge amounts of data,
they have to process this on the cluster.
Which is a cluster of server computers that will--
that are like way higher tech than anything
you could purchase individually.
But, for the sake of implementing a small ML
project, for the sake of learning about ML,
or for the sake of doing a fun small project, such as like attempting
to write an image pro-- like, recognitions
set, with just a bunch of images, this is something
that is very doable on most machines.
Of course, it really depends on like--
yeah, it will be software-- or hardware dependent here,
but I would say that short of being very, very old hardware,
then you should be able to at least get the script working.
But, of course, if it doesn't, cloud computing is always an option.
Google Colab is free, which is great, so it's--
and it's honestly not any different from just accessing the [? IDE. ?]
And, so, we really recommend that if you are
interested in things that will run really slowly, that you
look into cloud on Google Colab.
SPEAKER: OK, [? Aviral, ?] "Is it possible
to use two types of different algorithms at the same time
to increase the accuracy of the model?"
ZAD CHIN: It's not two possible models at the same time.
I think it is--
normally like we actually use different model,
and different model has its own strengths and weaknesses, I would say.
So, for example, like KNN, it might overfit very well.
But, then, I think, for example, like mo-- the example that you
have, KNN versus logistic regression, it's
not like we can integrate both two together.
I think both KNN and logistic regression has its own advantages
and disadvantages.
The idea is to test it against different kinds of ML models
that's available out there.
And to increase-- to see which in ML performs better on the current data
set.
So for example you can see that KNN performed really
well on the data set that you have just now,
but then it doesn't mean that KNN will always perform better on other data
sets that you are testing.
So it's actually very recommended to use different data sets.
But on the idea of integrating two machine learning models together
to get higher accuracy, it might be possible,
but I'm not really sure about it.
KEVIN XU: I would say the close analogy is that if you can recall-- well
actually, let's just go back to it.
If you can recall the nodes that we have, right?
You can think of each set of nodes as like, a different transformation
that you do to the data.
So what is often the case is that your data will go through many, many layers,
especially for things that are very complicated.
And, so, this is not necessarily applying two different machine learning
models, but you are--
this kind of transformation is very iterative.
And, so, this is like--
we call them layers, so like you pass the data through one layer,
and then you pass it through another layer, iteratively, until you get
to this end output layer.
So this actually goes into the specific of how the function works,
but you will often have to make multiple layers
to get good results for your data set, because the data is always complicated.
SPEAKER: This is, an I--
please excuse my pronunciation here, [? Effie ?]
writes, "Does ML help in cybersecurity?"
ZAD CHIN: Wait, the question is, "Does ML affect cyber security?
KEVIN XU: Or does it--
SPEAKER: "Does ML help in cybersecurity?"
ZAD CHIN: Oh, I think it does.
So, for example, I think I heard a lot of information
about how banks actually use ML to detect fraud transactions.
So in terms of that kind of way, there's a lot
of ways whereby ML actually helps in trying to prevent cybersecurity attacks
or like detect cybersecurity attacks, especially in a large company.
Because it's actually really well in trying
to find patterns, or unusual patterns in a huge amount of data set.
So I definitely think that it has a very huge application in cybersecurity
itself.
KEVIN XU: Yeah, and if there's one takeaway
that we want you to have from this, it's that the goal of-- what
ML is really good at doing is taking a lot of data
and finding connections between those data points.
So anything you can think of that needs to process a huge amount of data,
you can almost always apply some form of ML.
If this data is meaningful, of course.
So in terms of cybersecurity, like actually
building firewalls and those kinds of things,
it doesn't have to necessarily do as much
with processing huge amounts of data.
But, what, as Zad said, right, with like, fraudulent account
accesses, and things like that, that is something
that is totally very applicable to ML.
SPEAKER: OK.
[? Madhi ?] asks, "What is the AI application in video games nowadays?
And as a machine learning developer, can I work in game field?
KEVIN XU: Oh yeah, so, I can take this question.
Yeah, so, I don't know if you've seen, but the I think the team, OpenAI,
I think is run by Tesla, or is like a subsidiary or maybe--
ZAD CHIN: I think it's Google.
KEVIN XU: It might be Google, yeah.
So OpenAI is a team that has been developing machine learning
game software for quite a while.
And a couple of years ago, they released, like, big news, Dota 2, which
is one of the popular MOBA games, they managed
to write an AI that beat a team of professionals.
So this is totally something that is doable,
and currently, and I recommend you check out the OpenAI--
the stuff that they've been doing.
It's like, really cool and it's definitely very marketable
if you're interested in that kind of stuff.
ZAD CHIN: I think in terms of games, like DeepMind also created AlphaGo,
which is like one of--
I don't know whether that's considered a video game, but it's a game.
It's just super huge at that time, because it's the world's best Go
player on planet Earth.
It was like AlphaGo versus human.
There's also a movie about it, called Go, or AlphaGo, something like that.
Yeah, highly recommended.
Super--
KEVIN XU: Yes, so, furthermore, like in terms of not just video games,
people are still developing things to--
just test what we can use this for, in terms of not just these--
so, games have different classifications,
and Carnegie Mellon actually solved poker a few years ago.
Which is crazy, because poker, you don't always have all the information.
So there's so much more extrapolation that you
need to do from a given subset of information.
And, so, it's actually very interesting how far that machine learning
has come, from being able to take such a small, comparatively small set of data,
and be able to generalize it to big things.
SPEAKER: OK, and [? Yashvi ?] asks, "Just as you mentioned,
we trained the computer for game 1,000 times.
Was that the training part or the testing part from train-test?"
KEVIN XU: Ah, yeah, great.
So, when we did reinforcement learning, the training over 1,000 times
was simply to train the computer.
When you're reinforcement learning, your testing part,
you don't really have a test set, as opposed
to when you have a big data set of images where you can divide 80% of them
to train it, and then verify that your algorithm, or your regression
is correct with the remainder 20%.
In the case of the game, you just have-- like, once you've trained your AI,
you just have your AI play the game.
In which case, this is your testing phase.
It's like, did the AI win?
Like in our case, it was like, how many points did the AI get on average?
And, so, that is the testing phase, which doesn't actually
use a data set but rather just uses the game itself.
SPEAKER: OK.
"What is the--" This is from [? Madhi ?] again, "What is the application--"
Oh, no I already asked that one, sorry about that.
OK from HW, "Besides Google TensorFlow, what other ML platforms
should we look into?
Anything from IBM or Microsoft?"
I think you answered that one, right?
KEVIN XU: It's--
OK, so with the--
the thing about libraries, like, choosing the best library
can be very hard.
And this is like something that requires knowledge, and like,
you to just jump into ML and see what works and see what doesn't.
So, what we recommend, is that you don't concentrate so much
on optimizing, right now, everything.
You want to get something that works with the knowledge and the skills
that you have, first, before you can improve and try and optimize
to get algorithms that are very good for your data set and things like that.
So--
ZAD CHIN: I think the other thing about trying
to find an ML library is to find whether it is well documented
or whether actually a lot of people have tried it as well.
I personally don't know any library.
Maybe I'm just shallow minded, but I don't really know any library
from like IBM and Microsoft.
But I think they have a really strong IBM and Microsoft Research team.
Which I really admire as well.
But in terms of TensorFlow, they are really, really well documented.
There's a lot of examples out there, a lot of Stack Overflow posts about that
which is actually very important.
But you actually face a bottleneck, right?
Because you need someone to help you, someone to understand your data set.
And everybody is using it, it was really well documented.
So I would recommend TensorFlow if you are beginning,
but like, if you want to be more sophisticated,
you can start to build your own neural network.
And I think that will be the most sophisticated thing that you can do.
KEVIN XU: And of course, we don't expect you guys, in CS50,
to build your own right now.
But, in the future, just as consideration,
there's a lot of statistical theory that goes into what machine learning is.
And, so, there are Harvard classes, or MIT classes,
that you can take that are just totally on ML and applying it.
And if you are interested in talking about that,
feel free to reach out to one of us, and we'd be happy to chat with you.
ZAD CHIN: Yeah.
And, so, before we actually end, please take one minute to actually fill out
this feedback form, which is tinyurl.com/ML50-feedback.
And we actually really thank you so much for actually coming,
it's really late, or really early, I don't know what's the time zone.
But thank you so much for actually coming,
we actually enjoyed this great seminar, and we are very excited to share this.
And we hope that you learned something too.
So--
KEVIN XU: So, yeah, I think we'll be ending the recording now,
but might stay a little bit after to answer any further questions.