SPEAKER 1: All right.
Well, this is a CS50 Tech Talk.
Thank you all so much for coming.
So about a week ago, we circulated the Google Form,
as you might have seen, at 10:52 AM.
And by, like, 11:52 AM, we had 100 RSVP's,
which I think is sort of testament to just how much interest there
is in this world of AI and OpenAI and GPT, ChatGPT and the like.
And in fact, if you're sort of generally familiar with what everyone's
talking about but you haven't tried it yourself,
like, this is the URL at which you can try out this tool that you've probably
heard about, ChatGPT.
You can sign up for a free account there and start tinkering
with what everyone else has been tinkering with.
And then if you're more of the app-minded type, which you probably
are if you are here with us today, OpenAI, in particular,
has its own low-level APIs via which you can integrate AI
into your own software.
But of course, as is the case in computer science,
there's all the more abstractions and services
that have been built on top of these technologies.
And we're so happy today to be joined by our friends
from McGill University and Steamship, Sil and Ted,
from whom you'll hear in just a moment, to speak to us about how
they are making it easier to build, to deploy, to share applications using
some of these very same technologies.
So our thanks to them for hosting today, our friends at Plympton,
Jenny Lee, an alumna who's here with us today.
But without further ado, allow me to turn things over to Ted and Sil.
And pizza will be served shortly after 1:00 PM outside.
All right, over to you, Ted.
TED BENSON: Thanks a lot.
Hey, everybody.
It's great to be here.
I think we've got a really good talk for you today.
Sil is going to provide some research grounding into how it all works,
what's going inside the brain of GPT, as well as other language models.
And then I'll show you some examples that we're
seeing on the ground of how people are building apps
and what apps tend to work in the real world.
So our perspective is we're building AWS for AI apps.
So we get to talk to a lot of the makers who are building and deploying
their apps, and through that, see both the experimental end of the spectrum
and also see what kinds of apps are getting
pushed out there and turned into companies, turned into side projects.
We did a cool hackathon yesterday.
Many thanks to Neiman, to David Malan and CS50 for helping us
put all of this together, to Harvard for hosting it.
And there were two sessions.
Lots of folks built things.
If you go to steamship.com/hackathon, you'll find a lot of guides,
a lot of projects that people built. And you can follow along.
We have a text guide, as well.
Just as a quick plug for that, if you want to do it remotely or on your own.
So to tee up Sil, we're going to talk about basically two things
today that I hope you'll walk away with and really know how to then use
as you develop and as you tinker.
One is what is GPT and how is it working,
get a good sense of what's going on inside of it,
other than as just this magical machine that predicts things.
And then two is how are people building with it, and then, importantly,
how can I build with it, too, if you are a developer.
And if you have CS50 background, you should
be able to pick things up and start building some great apps.
I've already met some of the CS50 grads yesterday,
and the things that they were doing were pretty amazing.
So I hope this is useful.
I'm going to kick it over to Sil and talk about some
of the theoretical background of GPT.
SIL HAMILTON: Yeah.
So thank you, Ted.
My name is Sil.
I'm a graduate student in the digital humanities at McGill.
I study literature and computer science and linguistics in the same breath,
and I've published some research over the last couple of years
exploring what is possible with language models and culture, in particular.
And my half, or whatever, of the presentation
is to describe to you what is GPT.
That's really difficult to explain in 15 minutes,
and there are even a lot of things that we don't know.
But a good way to approach that is the first
consider all the things that people call GPT by, or descriptors.
So you can call them large language models.
You can call them universal approximators.
From computer science, you can say that it is a generative AI.
We know that they are neural networks.
We know that it is an artificial intelligence.
To some, it's a simulator of culture.
To others, it just predicts text.
It's also a writing assistant.
If you've ever used ChatGPT, you can plug in a bit of your essay,
get some feedback.
It's amazing for that.
It's a content generator.
People use it to do copywriting with Jasper.ai, Sudowrite, et cetera.
It's an agent.
So the really hot thing right now, if you might have
seen on Twitter, AutoGPT, Baby AGI.
People are giving these things tools and letting
them run a little bit free in the wild to interact with the world, computers,
et cetera.
We use them as chat bots, obviously.
And the actual architecture is a transformer.
So there's lots of ways to describe GPT, and any one of them
is a really perfectly adequate way to begin the conversation.
But for our purposes, we can think of it as a large language model,
and more specifically, a language model.
And a language model is a model of language,
if you'll allow me the tautology.
But really, what it does is it produces a probability
distribution over some vocabulary.
So let us imagine that we had the task of predicting
the next word of the sequence "I am."
So if I give a neural network the words "I am," what, of all words in English,
is the next most likely word to follow?
That, at its very core, is what GPT is trained to answer.
And how it does it is it has a vocabulary of 50,000 words,
and it knows roughly, given the entire internet,
which words are likely to follow other words of those 50,000 in some sequence,
up to 2,000 words, up to 4,000, up to 8,000, and now up to 32,000 in GPT-4.
So you give it a sequence.
Here, "I am."
And over the vocabulary of 50,000 words, it
gives you the likelihood of every single word that follows.
So here, it's "I am."
Perhaps the word "happy" is fairly frequent,
so we'll give that a high probability if we
look at all words, all utterances of English.
It might be "I am sad."
Maybe that's a little bit less probable.
"I am school."
That really should be at the end because I don't
think anybody would ever say that.
"I am Bjork."
That's a little bit--
it's not very probable.
It's less probable than happy/sad, but there's still
some probability attached to it.
And when we say it's probable, that's literally a percentage.
That's, like, "happy" follows "I am" maybe like 5% of the time.
"Sad" follows "I am" maybe 2% of the time, or whatever.
So for every word that we give GPT, it tries
to predict what the next word is across 50,000 words.
And it gives every single one of those 50,000 words
a number that reflects how probable it is.
And the really magical thing that happens is you can generate new texts.
So if you give GPT "I am" and it predicts
"happy" as being the most probable word over 50,000, you can then append it to
"I am."
So now you say "I am happy."
And you feed it into the model again.
You sample another word.
You feed it into the model again, and again and again and again.
And there's lots of different ways that "I am happy," "I am sad" can go.
And you add a little bit of randomness, and all of a sudden,
you have a language model that can write essays, that can talk,
and a whole lot of things, which is really unexpected
and something that we didn't predict even five years ago.
So this is all relevant.
And if we move on, as we scale up the model and we give it more compute,
in 2012, AlexNet came out, and we figured out we can give the model--
we can run the model in GPUs.
So we can speed up the process.
We can give the model lots of information
downloaded from the internet, and it learns more and more and more.
And the probabilities that it gives you get
better as it sees more examples of English on the internet.
So we have to train the model to be really large, really wide,
and we have to train it for a really long time.
And as we do that, the model gets more and more better and expressive
and capable, and it also gets a little bit intelligent,
and for reasons we don't understand.
But also, the issue is that because it learns to replicate the internet,
it knows how to speak in a lot of different genres of text
and a lot of different registers.
If you begin the conversation like, "ChatGPT,
can you explain the moon landing to a six-year-old in a few sentences,"
GPT-3--
this is an example drawn from the InstructGPT paper from OpenAI--
GPT-3 would have just been like, "OK, so you're giving me an example,
like explain the moon landing to a six-year-old.
I'm going to give you a whole bunch of similar things
because those seem very likely to come in a sequence."
It doesn't necessarily understand that it's being asked a question
and has to respond with an answer.
GPT-3 did not have that apparatus that interfaced for responding to questions.
And the scientists at OpenAI came up with a solution.
And that's let's give it a whole bunch of examples of question and answers
such that we first train it on the internet,
and then we train it with a host of questions and answers
such that it has the knowledge of the internet,
but really knows that it has to be answering questions.
And that is when ChatGPT was born.
And that's when it gained 100 million users in one month.
I think it beat TikTok's record of 20 million in one month.
It was a huge thing.
And for a lot of people, they went, "oh, this thing is intelligent.
I can ask it questions.
It answers back.
We can work together to come to a solution."
And that's because it's still predicting words, it's still a language model,
but it knows to predict words in the framework of a question and answer.
So that's what a prompt is.
That's what instruction tuning is.
That's a key word.
That's what RLHF is, if you've ever seen that acronym, reinforcement
alignment with human feedback.
And all of those combined means that the models that are coming out today,
the types of language predictors that are coming out today
work to operate in a Q&A form.
GPT-4 exclusively only has the aligned model available.
And this is a really great, solid foundation
to build on because you can do all sorts of things.
You can ask it, "ChatGPT, can you do this for me?
Can you do that for me?"
You might have seen that OpenAI has allowed plug-in access to ChatGPT.
So it can access Wolfram.
It can search the web.
It can do Instacart for you.
It can look up recipes.
Once the model knows that not only it has to predict language,
but that it has to solve a problem--
and the problem here being give me a good answer to my question--
it's suddenly able to interface with the world in a really solid way.
And from there on, there's been all sorts
of tools that build on this Q&A form that ChatGPT uses.
You have AutoGPT.
You have LangChain.
You have React.
There was a React paper where a lot of these come from.
And turning the model into an agent with which to achieve any ambiguous goal
is where the future is going.
And this is all thanks to instruction tuning.
And with that, I think I will hand it off
to Ted, who will be giving a demo, or something along those lines,
for how to use GPT as an agent.
So.
TED BENSON: All right, so I'm a super applied guy.
I kind of look at things and think, OK, how can I add this LEGO, add that LEGO,
and clip them together and build something with it.
And right now, if you look back in computer science history, when
you look at the kinds of things that were being done in 1970,
right after computing was invented, the microprocessors were invented,
people were doing research like how do I sort a list of numbers.
And that was meaningful work, and importantly,
it was work that's accessible to everybody
because nobody knows what we can build with this new kind of oil,
this new kind of electricity, this new kind of unit of computation
we've created.
And anything was game, and anybody could participate
in that game to figure it out.
And I think one of the really exciting things about GPT right now is, yes,
in and of itself, it's amazing.
But then, what could we do with it if we call it over and over again,
if we build it into our algorithms and start
to build it into broader software?
So the world really is yours to figure out
these fundamental questions about what could you
do if you could script computation itself over and over again in the way
that computers can do.
Not just talk with it, but build things atop it.
So we're a hosting company.
We host apps.
And these are just some of the things that we see.
I'm going to show you demos of this with code
and try to explain some of the thought process.
But I wanted to give you a high-level overview
of you've probably seen these on Twitter,
but kind of when it all sorts out to the top, these are some of the things
that we're seeing built and deployed with language models today.
Companionship.
That's everything from I need a friend to I need a friend with a purpose.
I want a coach.
I want somebody to tell me, "go to the gym and do these exercises."
I want somebody to help me study a foreign language.
Question answering.
This is a big one.
This is everything from your newsroom having a Slack bot that
helps assist you, does this article conform
to the style guidelines of our newsroom, all the way through to I
need help on my homework, or hey, I have some questions
that I want you to ask Wikipedia, combine it with something else,
synthesize the answer, and give it to me.
Utility functions.
I would describe this as there's a large set of things for which
human beings can do them if only--
or computers could do them if only they had access
to language computation, language knowledge.
An example of this would be read every tweet on Twitter.
Tell me the ones I should read.
That way, I only get to read the ones that actually make sense to me
and I don't have to skim through the rest.
Creativity.
Image generation, text generation, storytelling,
proposing other ways to do things.
And then these wild experiments in kind of Baby AGI,
as people are calling them, in which the AI itself decides what to do
and is self-directed.
So I'll show you examples of many of these and what the code looks like.
And if I were you, I would think about these
as categories within which to both think about what you might build
and then also seek out starter projects for how you
might go about building them online.
All right.
So I'm just going to dive straight into demos and code for some of these
because I know that's what's interesting to see as fellow builders,
with a high-level diagram for some of these as to how it works.
So approximately, you can think of a companionship bot
as a friend that has a purpose to you.
And there are many ways to build all of these things,
but one of the ways you can build this is simply
to wrap GPT or a language model in an endpoint that additionally injects
into the prompt some particular perspective or some particular goal
that you want to use.
It really is that easy, in a way, but it's also very hard
because you need to iterate and engineer the prompt so that it consistently
performs the way you want it to perform.
So a good example of this is something somebody built in the hackathon
yesterday.
And I just wanted to show you the project that they built.
It was a Mandarin idiom coach.
And I'll show you what the code looked like first.
I'll show you the demo first.
I think I already pulled it up.
Here we go.
So the buddy that this person wanted to create
was a friend that, if you gave it a particular problem
you were having, it would pick a Chinese idiom,
a four-character chengyu that described, poetically, like,
here's a particular way you could say this,
and it would tell it to her, so that the person who built this
was studying Chinese and she wanted to learn more about it.
So I might say something like, "I'm feeling very sad."
And it would think a little bit.
And if everything's up and running, it will
generate one of these four-character phrases
and it will respond to it with an example.
Now, I don't know if this is correct or not.
So if somebody can call me out if this is actually incorrect,
please call me out.
And it will then finish up with something encouraging,
saying, "hey, you can do it.
I know this is hard.
Keep going."
So let me show you how they built this.
And I pulled up the code right here.
So this was the particular starter Replit
that folks were using in the hackathon yesterday.
And we pulled things up into basically you have a wrapper around GPT.
And there's many things you could do, but we're
going to make it easy for you to do two things.
One of them is to inject some personality into the prompt.
And I'll explain what that prompt is in a second.
And then the second is add tools that might
go out and do a particular thing-- search the web or generate an image
or add something to a database or fetch something from a database.
So having done that, now you have something more than GPT.
Now you have GPT, which we all know what it is and how we can interact with it,
but you've also added a particular lens through which it's talking
to you and, potentially, some tools.
So this particular Chinese tutor, all it took to build that was four lines.
So here's a question that I think is frying the minds of everybody
in the industry right now.
So is this something that we'll all do casually?
And nobody really knows.
Will we just all say in the future to the LLM, "hey, for the next five
minutes, please talk like a teacher?"
Maybe.
But also, definitely in the meantime and maybe in the future,
it makes sense to wrap up these personalized endpoints
so that when I'm talking to GPT, I'm not just talking to GPT.
I have a whole army of different buddies,
of different companions that I can talk to.
They're kind of human and kind of talk to me interactively,
but because I preloaded them with, "hey, by the way, you particular,
I want you to be a kind, helpful Chinese teacher that
responds to every situation by explaining the chengyu that fits it.
Speak in English and explain the chengyu and its meaning.
Then provide a note of encouragement about learning language."
And so just adding something like that, even if you're a non-programmer,
you can just type deploy and it'll pop it up to the web.
It'll take it over to a Telegram bot that you can even interact with.
"Hey, I'm feeling too busy."
And interact with it over Telegram, over the web.
And this is the kind of thing that's now within reach for everybody from a CS101
grad, so I'm using the general purpose framing,
all the way through to professionals in the industry
that you can do just with a little bit of manipulation
on top of this raw unit of conversation and intelligence.
So companionship is one of the first common types of apps that we're seeing.
So a second kind of app that we're seeing-- and this blew up--
for those of you who are kind of Twitter followers,
this blew up I think the last few months, is question answering.
And I want to unpack a couple of different ways
this can work because I know many of you have probably already tried
to build some of these kinds of apps.
There's a couple of different ways that it works.
The general framework is a user queries GPT.
And maybe it has general-purpose knowledge.
Maybe it doesn't have general-purpose knowledge.
But what you want it to say back to you is something specific about an article
you wrote, or something specific about your course syllabus,
or something specific about a particular set of documents from the United
Nations on a particular topic.
And so what you're really seeking is what we all hoped
the customer service bot would be.
Like, we've all interacted with these customer service bots,
and we're kind of smashing our heads on the keyboard as we do it.
But pretty soon, we're going to start to see very high-fidelity
bots that interact with us comfortably.
And this is approximately how to do it as an engineer.
So here's your game plan as an engineer.
Step one, take the documents that you want it to respond to.
Step two, cut them up.
Now, if you're an engineer, this is going to madden you.
You don't cut them up in a way that you would hope.
For example, you could cut them up into clean sentences
or clean paragraphs or semantically coherent sections.
And that would be really nice.
Honestly, the way that most folks do it--
and this is a simplification that tends to be just fine--
is you window, you have a sliding window that goes over the document,
and you just pull out fragments of text.
Having pulled out those fragments of text,
you turn them into something called an embedding vector.
So an embedding vector is a list of numbers
that approximate some point of meaning.
So you've already all dealt with embedding vectors yourself
in regular life.
And the reason you have, and I know you have,
is because everybody's ordered food from Yelp before.
So when you order food from Yelp, you look at what genre of restaurant is it.
Is it a pizza restaurant?
Is it an Italian restaurant?
Is it a Korean barbecue place?
You look at how many stars does it have-- one, two, three, four, five.
You look at where is it.
So all of these you can think of as points in space, dimensions in space.
Korean barbecue restaurant, four stars, near my house.
It's a three-number vector.
That's all this is.
So this is a 1,000 number vector or a 10,000 number vector.
Different models produce different size vectors.
All it is, is chunking pieces of text, turning it
into a vector that approximates meaning, and then you put it in something
called a vector database.
And a vector database is just a database that stores numbers.
But having that database, now when I ask a question, I can search the database
and I can say, "hey, the question was, what does CS50 teach?"
What pieces of text in the database have vectors similar to the question,
what does CS50 teach?
And there's all sorts of tricks and empires
being made on refinements of this general approach.
But at the end, you, the developer, model it simply as thus.
And then when you have your query, you embed it,
you find the document fragments, and then you put them into a prompt.
And now we're just back to the personality, the companionship bot.
Now it's just a prompt.
And the prompt is, "you're an expert in answering questions.
Please answer user-provided question using source documents,
results from the database."
That's it.
So after all of these decades of engineering of these customer service
bots, it turns out, with a couple of lines of code, you can build this.
So let me show you.
I made one just before the class with the CS50 syllabus.
So we can pull that up.
And I can say I added the PDF right here.
So I just searched--
I apologize, I don't know if it's an accurate or recent syllabus.
I just searched the web for CS50 syllabus PDF.
I put the URL in here.
It loaded it into here.
This is just like a 100-line piece of code
deployed that will now let me talk to it.
And I can say, "what will CS50 teach me?"
So under the hood now, what's happening is exactly what that slide just
showed you.
It takes that question, "What will CS50 teach me."
It turns it into a vector.
That vector approximates, without exactly representing,
the meaning of that question.
It looks into a vector database that Steamship
hosts of fragments from that PDF.
And then it pulls out a document and then passes it
to a prompt that says, "hey, you're an expert at answering questions.
Someone has asked you what does CS50 teach.
Please answer it using only the source documents and source materials
I've provided."
Now, those source materials are dynamically loaded into the prompt.
It's just basic prompt engineering.
And I want to keep harping back onto that.
What's amazing about right now as builders is that so many things just
boil down to very creative, tactical rearrangement of prompts,
and then using those over and over again in an algorithm
and putting that into software.
So the result-- and again, it could be lying, it could be making things up,
it could be hallucinating-- is "CS50 will teach students how to think
algorithmically and solve problems efficiently,
focusing on topics such as abstraction," da-da-da-da-da.
And then it returns the source document from which it was found.
So this is another big category of which there
are tons of potential applications because you
can repeat for each context.
You can create arbitrarily many of these once it's software,
because once it's software, you can just repeat it over and over again.
So for your dorm, for your club, for your Slack, for your Telegram.
You can start to begin putting pieces of information
in and then responding to it.
And it doesn't have to be documents.
You can also load it straight into the prompt.
I think I have it pulled up here.
And if I don't, I'll just skip it.
Oh, here we go.
One other way you can do question answering,
because I think it's healthy to always encourage
the simplest possible approach to something,
you don't need to engineer this giant system.
It's great to have a database.
It's great to use embeddings.
It's great to use this big approach.
It's fancy.
It scales.
You can do a lot of things.
But you can also get away with a lot by just pushing it all into a prompt.
And as an engineer, [INAUDIBLE],, one of our teammates here always says,
"engineers should aspire to be lazy."
And I couldn't agree more.
You, as an engineer, should want to set yourself
up so that you can pursue the lazy path to something.
So here's how you might do the equivalent of a question answering
system with a prompt alone.
Let's say you have 30 friends.
And each friend is good at a particular thing.
Or you can-- this is isomorphic to many other problems.
You can simply just say, "hey, I know certain things.
Here's the things I know.
A user is going to ask me something.
How should we respond?"
And then you load that into an agent.
That agent has access to GPT.
You can ship-deploy it.
And now you've got a bot that you can connect to Telegram,
you can connect to Slack.
And that bot, now, it won't always give you
the right answer, because at a certain level,
we can't control the variance of the model underneath,
but it will tend to answer with respect to this list.
And the degree to which it tends is, to a certain extent, something
that both industry is working on to just give everybody
as a capacity, but also you, doing prompt engineering,
to tighten up the error bars on it.
So I'll show you just a few more examples.
And then in about eight minutes, I'll turn it over to questions,
because I'm sure you've got a lot about how to build things.
So just to give you a sense of where we are.
This is one, I don't have a demo for you, but if you were to come to me
and you were to say, "Ted, I want a weekend hustle, man.
What should I build?"
Holy moly.
There are a set of applications that I would describe as utility functions.
I don't like that name because it doesn't sound exciting,
and this is really exciting.
And it's low-hanging fruits that automate tasks that
require basic language understanding.
So examples for this are generate a unit test.
I don't know how many of you have ever been writing tests
and you're just like, "oh, come on.
I can get through this.
I can get through this."
If you're a person who likes writing tests, you're a lucky individual.
Looking up the documentation for a function, rewriting a function,
making something conform to your company guidelines, doing a brand check,
all of these things are things that are kind
of relatively context-free operations, or scoped-context operations
on a piece of information that requires linguistic understanding.
And really, you can think of them as something
that is now available to you as a software builder,
as a weekend project builder, as a startup builder.
And you just have to build the interface around it
and present it to other people in a context in which it's
meaningful for them to consume.
And so the space of this is extraordinary.
I mean, it's the space of all human endeavor, now with this new tool,
I think is the way to think about it.
People often joke about how, when you're building a company, when you're
building a project, you don't want to start with the hammer
because you want to start with the problem instead.
And it's generally true, but my God, we've
just got a really cool new hammer.
And to a certain extent, I would encourage you to at least casually,
on the weekends, run around and hit stuff with it
and see what can happen from a builder's, from a tinkerer's,
from an experimentalist's point of view.
And then the final one is creativity.
This is another huge mega app.
Now, I primarily live in the text world, and so I'm
going to talk about text-based things.
I think, so far, this has mostly been growing in the imagery world
because we're such visual creatures and the images you can generate
are just staggering with AI.
It certainly brings up a lot of questions,
too, around IP and artistic style.
But the template for this, if you're a builder, that we're seeing in the wild
is approximately the following.
And the thing I want to point out is domain knowledge here.
This is really the purpose of this slide,
is to touch on the importance of the domain knowledge.
So many people approximately find the creative process as follows.
Come up with a big idea.
Overgenerate possibilities.
Edit down what you overgenerated.
Repeat.
Right?
Anybody who's been a writer knows, when you write, you write way too much,
and then you have to delete lots of it.
And then you revise, and you write way too much,
and you have to delete lots of it.
This particular task is fantastic for AI.
One of the reasons it's fantastic for AI is
because it allows the AI to be wrong.
You know, you've preagreed you're going to delete lots of it.
And so if you preagree, "hey, I'm just going
to generate five possibilities of the story I might tell,
five possibilities of the advertising headline,
five possibilities of what I might write my thesis on,"
you preagreed it's OK if it's a little long because you are
going to be the editor that steps in.
And here's the thing that you really should bring to the table,
is don't think about this as a technical activity.
Think about this as your opportunity not to put GPT in charge.
Instead, for you to grasp the steering wheel tighter--
I think, at least--
in Python or the language you're using to program
because you have the domain knowledge to wield GPT in the generation of those.
So let me show you an example of what I mean by that.
So this is a cool app that someone created for the Writing Atlas Project.
So Writing Atlas is a set of short stories.
And you can think of it as Good Reads for short stories.
So you can go in here, you can browse different stories.
And this was something somebody created where you can type
in a story description that you like.
And this is going to take about a minute to generate,
so I'm going to talk while it's generating.
And while it's working, what it's doing--
and I'll show you the code in a second--
is it's searching through the collection of stories for similar stories.
And here's where the domain knowledge part comes in.
Then it uses GPT to look at what it was that you wanted
and use knowledge of how an editor, how a bookseller thinks
to generate a set of suggestions specifically
through the lens of that perspective, with the goal of writing
that beautiful handwritten note that we sometimes see in a local bookstore
tacked on underneath a book.
And so it doesn't just say, "hey, you might like this,
here's a general-purpose reason why you might like this,"
but specifically "here's why you might like this,"
with respect to what you gave it.
It's either stalling out or it's taking a long time.
Oh, there we go.
So here's its suggestions.
And in particular, these things, these are
things that only a human could know, at least for now.
Two humans, specifically, the human who said they wanted to read a story--
that's the text that came in-- and then the human
who added domain knowledge to script a sequence of interactions
with the language model so that you could provide
very targeted reasoning over something that
was informed by that domain knowledge.
So for these utility apps, bring your domain knowledge.
Let me actually show you how this looks in code
because I think it's useful to see how simple and accessible this is.
This is really a set of prompts.
So why might they like a particular location?
Well, here's the prompt that did that.
This is an open source project.
And it has a bunch of examples, and then it
says, well, here's the one that we're interested in.
Here's the audience.
Here's a couple of examples of why might people like a particular thing,
in terms of audience.
It's just another prompt.
Same for topic.
Same for explanation.
And if you go down here and look at how it was done, suggesting the story is--
what is this, line 174 to line 203--
it really is-- and again, over and over again,
I want to impress upon you-- this really is within reach.
It's really just, what, 20 odd lines of step one, search
in the database for similar stories.
Step two, given that I have similar stories, pull out the data.
Step three, with my domain knowledge in Python, now run these prompts.
Step four, prepare that into an output.
So the thing we're scripting itself is some approximation of human cognition,
if you're willing to go there metaphorically.
We're not sure-- I'm not going to weigh in
on where we are in the is OpenAI a life form argument.
All right.
One kind of really far out there thing, and then I'll
tie it up for questions, because I know there's probably a lot.
And I also want to make sure you get great pizza in your bellies.
And that is Baby AGI, AutoGPT is what you might
have heard them called on Twitter.
I think of them as multi-step planning bots.
So everything I showed you so far was approximately one-shot interactions
with GPT.
So this is the user says they want something,
and then either Python mediates interactions with GPT or GPT
itself does some things with the inflection of a personality
that you've added from some prompt engineering.
Really useful.
Pretty easy to control.
If you want to go to production, if you want to build a weekend project,
if you want to build a company, that's a great way to do it right now.
This is wild.
And if you haven't seen this stuff on Twitter,
I would definitely recommend going to search for it.
This is what happens--
the simple way to put it is-- if you put GPT in a for loop.
If you let GPT talk to itself and then tell itself what to do.
So it's an emergent behavior.
And like all emergent behaviors, it starts with a few simple steps.
In Conway's Game of Life, many elements of reality
turn out to be math equations that fit on a t-shirt,
but then when you play them forward in time, they generate DNA.
They generate human life.
So this is approximately, step one, take a human objective.
Step two, your first task is to write yourself a list of steps.
And here's the critical part--
repeat.
Now do the list of steps.
Now, you have to embody your agent with the ability to do things.
So it's really only limited to do what you give it the tools to do
and what it has the skills to do.
So obviously, this is still very much a set
of experiments that are running right now.
But it's something that we'll see unfold over the coming years.
And this is the scenario in which Python stops
becoming so important because we've given it the ability to actually
self-direct what it's doing.
And then it finally gives you a result.
And I want to give you an example still of just, again, impressing upon you
how much of this is prompt engineering, which is wild, how little code this is.
Let me show you what Baby AGI looks like.
So here is a Baby AGI that you can connect to Telegram.
And this is an agent that has two tools.
So I haven't explained to you what an agent is.
I haven't explained to you what tools are.
I'll give you a quick, one-sentence description.
An agent is just a word to mean GPT plus some bigger body in which it's living.
Maybe that body has a personality.
Maybe it has tools.
Maybe it has Python mediating its experience with other things.
Tools are simply ways in which the agent can choose to do things.
Like, imagine if GPT could say, "order a pizza."
And instead of you seeing the text "order a pizza,"
that caused a pizza to be ordered.
That's a tool.
So these are two tools it has.
One tool is generate a to-do list.
One tool is do a search on the web.
And then down here, it has a prompt saying, "hey,
your goal is to build a task list and then do that task list."
And then this is just placed into a harness that does it over and over
again.
So after the next task, kind of on cue, the results of that task.
And keep it going.
And so in doing that, you get this kickstarted loop,
where, essentially, you kickstart it and then the agent is talking to itself.
Talking to itself.
So this, unless I'm wrong, I don't think this has yet reached production,
in terms of what we're seeing in the field of how
people are deploying software.
But if you want to dive into the wildest part of experimentation,
this is definitely one of the places you can start.
And it's really within reach.
All you have to do is download one of the starter projects for it.
And you can kind of see right in the prompting,
here's how you kickstart that process of iteration.
All right, so I know that was super high-level.
I hope it was useful.
It's, I think, from the field, from the bottoms up what we're seeing
and what people are building, kind of these high-level categories of apps
that people are making, all of these apps
are apps that are within reach to everybody,
which is really, really exciting.
And I suggest Twitter as a great place to hang out and build things.
There's a lot of AI builders on Twitter publishing.
And I think we've got a couple of minutes before pizza is arriving.
Maybe 10 minutes.
Keep on going?
Oh.
So if there's any questions, why don't we kick it to that, because I'm sure
there's some questions that you all have.
I guess ended a little early.
Yes.
AUDIENCE: Yeah, so I have a question about hallucinations.
And so when you're building these sorts of applications in the apps,
for example, let's say--
I'm giving you, like, a physics problem, from a [? pset, ?]
and we want to do that.
TED BENSON: Yeah.
AUDIENCE: And it's, 40% of the time, just wrong.
TED BENSON: Yeah.
AUDIENCE: Do you have any actionable recommendations
that developers should be doing to make it hallucinate any less?
Or maybe even things that OpenAI on the back end
should be doing to reduce hallucinations?
Would it be something where you use RLHF?
Any thoughts there?
TED BENSON: So the question was how-- approximately, how do you
manage the hallucination problem?
If you give it a physics lecture, and you ask it a question, on the one hand,
it appears to be answering you correctly.
On the other hand, it appears to be wrong to an expert's eye 40%
of the time, 70% of the time, 10% of the time.
It's a huge problem.
And then what are some ways as developers,
practically, you can use to mitigate that.
I'll give an answer.
Sil, you may have some specific things too.
So one high-level answer is the same thing
that makes these things capable of synthesizing
information is part of the reason why it hallucinates for you.
So it's hard to have your cake and eat it too to a certain extent.
So this is part of the game.
In fact, humans do it too.
People talk about just folks who are too aggressive in their assumptions
about knowledge--
I can't remember the name for that phenomenon--
where you'll just say stuff.
So we do it to.
Some things you can do are--
kind of a range of activities-- depending on how much money
you're willing to spend, how much technical expertise you have,
it can range from fine tuning a model, to practically--
I'm in the applied world, so I'm very much in a world of duct tape
and how developers get stuff done.
So some of the answers.
I'll give you are sort of very duct tapey answers.
Giving it examples tends to work for acute things.
If it's behaving in wild ways, the more examples you give it, the better.
That's not going to solve the domain of all of physics.
So for the domain of all of physics, I'm going to--
I'm going to bail and give it to you because I
think you are far more equipped than me to speak on that.
SIL HAMILTON: Sure, so the model doesn't have a ground truth.
It doesn't know anything.
Any sense of meaning that is derived from the training process
is purely out of differentiation.
One word is not another word.
Words are not used in the same context it understands everything
only through examples given through language.
It's like someone who learned English or how to speak,
but they grew up in a featureless, gray room.
They've never seen the outside world.
They have nothing to rest on that tells them that something is true
and something is not true.
So from the model's perspective, everything that it says is true.
It's trying its best to give you the best answer possible.
And if lying a little bit or conflating two different topics
is the best way to achieve that, then it will decide to do so.
It's a part of the architecture.
We can't get around it.
There are a number of cheap tricks that surprisingly get
it to confabulate or hallucinate less.
One of them includes-- recently, there was a paper that's a little funny.
If you get it to prepend to its answer, "My best guess is,"
that will actually improve or reduce hallucinations by about 80%.
So clearly, it has some sense that some things are true
and other things are not, but we're not quite sure what that is.
To add on to what Ted was saying, a few cheap things you can do,
include letting it Google, or Bing-- as in Bing Chat, what they're
doing-- it cites this information.
Asking it to make sure its own response is good.
If you've ever had ChatGPT generate a program--
there's some kind of problem, and you ask ChatGPT--
I think there's a mistake.
Often, it'll locate the mistake itself.
Why it didn't produce the right answer at the very beginning,
we're still not sure, but we're moving in the direction
of reducing hallucinations.
Now with respect to physics, you're going
to have to give it an external database to rest on because internally
for really, really domain-specific knowledge,
it's not going to be as deterministic as one would like.
These things work in continuous spaces.
These things, they don't know what is wrong, what is true.
And as a result, we have to give it tools.
So everything that Ted demoed today is really
striving at reducing hallucinations, actually, really, and giving it
more abilities.
I hope that answers your question.
TED BENSON: One of the ways too--
I'm a simple guy.
I tend to think that all of the world tends to be just a few things
repeated over and over again.
And we have human systems for this.
In a team, like companies work-- or a team playing sport,
and we're not right all the time, even when we aspire to be.
And so we have systems that we've developed as humans
to deal with things that may be wrong.
So human number one proposes an answer.
Human number two checks their work.
Human number three provides the final sign off.
This is really common.
Anybody who's worked in a company has seen this in practice.
The interesting thing about the state of software right now,
we tend to be in this mode in which we're just
talking to GPT as one entity.
But once we start thinking in terms of teams, so to speak,
where each team member is its own agent with its own set of objectives
and skills, I suspect we're going to start
seeing a programming model in which the way to solve this
might not necessarily be make a single brain smarter, but instead be draw
upon the collective intelligence of multiple software agents,
each playing a role.
And I think that that would certainly follow the human pattern of how
we deal with this.
SIL HAMILTON: To give it an analogy, space shuttles,
things that go into space, spacecraft, they have to be good.
If they're not good, people die.
They have no margin for error at all.
And as a result, we overengineer in those systems.
Most spacecraft have three computers.
And they all have to agree in unison on a particular step to go forward.
If one does not agree, then they recalculate, they recalculate,
they recalculate until they arrive at something.
But the good thing is that hallucinations are generally not
a systemic problem in terms of its knowledge.
It's often a one-off.
The model or something tripped it up, and it just produced a hallucination
in that one instance.
So if there's three models working in unison, just as Ted is saying,
that will, generally speaking, improve your success.
Yes, sir.
AUDIENCE: A number of the examples you show how assertions like
you are an engineer, you aren't AI.
You are a teacher.
TED BENSON: Yeah.
AUDIENCE: What's the mechanism by which that influences
this computation of probabilities?
TED BENSON: Sure.
I'm going to give you what might be an unsatisfying answer, which
is it tends to work.
But I think we know why it tends to work.
And again, it's because these language models
approximate how we talk to each other.
So if I were to say to you-- hey, help me out.
I need you to mock interview me.
That's a direct statement I can make that kicks you
into a certain mode of interaction.
Or if I say to you-- help me out.
I'm trying to apologize to my wife.
She's really mad at me.
Can you role play with me?
That kicks you into another mode of interaction.
And so it's really just a shorthand that people
have found to kick the agent in-- to kick
the LLM in to a certain mode of interaction that
tends to work in the way that I, as a software developer,
am hoping it would work.
SIL HAMILTON: And to really quickly add on to that.
Being in the digital humanities that I am,
I like to think of it as a narrative.
A narrative will have a few different characters talking to each other.
Their roles are clearly defined.
Two people are not the same.
This interaction with GPT, it assumes a personality.
It can simulate personalities.
It, itself, is not conscious in any way, but it can certainly
predict what a conscious being would react like in a particular situation.
So when we're going URX, it is drawing up that personality
and talking as though it is that person.
Because it is like completing a transcript
or completing a story in which that character is present, and interacting,
and is active.
So, yeah.
TED BENSON: I think we got about five minutes until the pizza outside.
SPEAKER 1: Eight minutes.
TED BENSON: Eight minutes.
Yes, sir.
AUDIENCE: So I'm not a VS person, but it's been fun playing with this.
And I understand the word by word generation and the vibe.
The feeling of it in the narrative.
Some of my friends and I have tried giving it
logic problems, like things from the LSAT, for example, and it doesn't work.
And I'm just wondering why that would be.
So it will generate answers that sound very plausible
rhetorically-- like given this condition, x given this will be y,
but it will often even contradict itself in its answers.
But it's almost never correct.
So I was wondering why that would be?
Like, it just can't reason?
It can't think?
And can you-- would we get to a place where it can, so to speak?
I mean not-- you know what I mean, I don't mean to think like
it's conscious, I mean have thoughts--
[INAUDIBLE]?
TED BENSON: You want to about react?
AUDIENCE: I don't know how else to say that.
TED BENSON: So GPT4--
when GPT4 released back in March, I think it was, it was passing LSAT.
AUDIENCE: It was?
TED BENSON: It was, yeah.
AUDIENCE: [INAUDIBLE]
TED BENSON: Yes.
AUDIENCE: [INAUDIBLE]
TED BENSON: Yes, it just passed, as I understand it.
AUDIENCE: Maybe it's because we're not using [INAUDIBLE]..
TED BENSON: That's one of the weird things, is that--
AUDIENCE: ChatGPT.
TED BENSON: Yeah--
AUDIENCE: [INAUDIBLE]
TED BENSON: If you pay for ChatGPT, they give you access to the better model.
And one of the interesting things with it is prompting.
It's so finicky.
It's very sensitive to the way that you prompt.
Earlier on when, when GPT3 came out, some people were going, look,
it can pass literacy tests, or no, it can't pass literacy tests.
And then people who were pro or anti GPT would be like,
I modified the prompt a little bit, suddenly it can or suddenly it can't.
These things are not conscious.
Their ability to reason is like an alien's.
They're not us.
They don't think like people.
They're not human.
But they certainly are capable of passing some things empirically, which
demonstrates some sort of rationale or logic within the model.
But we're still slowly figuring out, like a prompt whisperer,
what exactly the right approach is.
AUDIENCE: Obviously, having GPT running and prompting it continuously
is very expensive in terms of GPU.
How do you see instances where it pre-creates some sort of business value
in a startup or a company?
Was there a real added value having these little AI apps
in terms of [INAUDIBLE]?
TED BENSON: Yeah, we host companies on top of us,
who that's their primary product.
The value that it adds is, like any company--
I mean, it's what is the Y Combinator motto--
"Make something people want."
I mean, I wouldn't think of this as GPT inherently provides value for you
as a builder.
That's their product.
That's OpenAI's product.
You pay ChatGPT for prioritized access.
Where your product might be is how you take that and combine it
with your data, somebody else's data, some domain knowledge, some interface
that then helps apply it to something.
Two things are both true.
There are a lot of experiments going on right now,
both for fun and people trying to figure out where the economic value is.
But folks are also spinning up companies that are 100% supported
by applying this to data.
AUDIENCE: OK.
For a company that wouldn't have--
wouldn't be AI focused [INAUDIBLE],, just using or developing
in-house apps that use GPT for productivity.
TED BENSON: I think that it is likely that today we call this GPT,
and today we call these LLMs, and tomorrow it
will just slide into the ether.
Imagine what the-- imagine what the progression is going to be.
Today, there's one of these that people are primarily playing with.
There's many of them that exist, but one people are primarily building on top.
Tomorrow, we can expect that there will be many of them.
And the day after that, we can expect they're going to be on our phones,
and they're not even going to be connected to the internet.
And for that reason, I think that-- like today,
we don't call our software microprocessor tools or microprocessor
apps, like the processor just exists--
I think that one useful model, five years out,
10 years out is to-- even if it's only metaphorically true and not literally
true--
I think it's useful to think if this as a second processor.
We had this before with floating point co-processors and graphics
co-processors already, as recently as the '90s, where
it's useful to think of the trajectory of this as just another thing
that computers do, can do, and it will be incorporated
into absolutely everything.
SIL HAMILTON: Hence, the term foundation model, which also crops up.
So the pizza's ready?
One more question.
TED BENSON: Maybe one more and then we'll break for some food.
In the glasses right there.
AUDIENCE: Do you have recommendations for the--
TED BENSON: Sorry, I was just being told we need to get two more.
AUDIENCE: Do you have any recommendations for how
ChatGPT will [INAUDIBLE] structure of data, like JSON, for example,
[INAUDIBLE].
TED BENSON: It's hard to get it to do that reliably.
It's incredibly useful to get it to do reliably.
So some tricks you can use are you can give it examples.
You can just ask it directly.
Those are two common tricks.
And look at the prompts that others have used to work.
There's a lot of art to finding the right prompt right now.
A lot of it is magic incantation.
Another thing you can do is post process it so that you can do some checking,
and you can have a happy path, in which it's a one shot,
and you get your answer, and then a sad path, in which maybe you
fall back on other prompts.
So then you're going for the diversity of approach,
where it's fast by default, it's slow, but ultimately
converging upon higher likelihood of success if it fails.
And then something that I'm sure we'll see and people do later on
is fine tune-- instruction tuning style models,
which are more likely to respond with a computer parsable output.
I guess one last question.
AUDIENCE: Sure, so one, you talked-- a couple of things.
One is this you talk about domain expertise here.
And you're encoding a bunch of domain expertise in terms of the prompts
that you're loading there.
What is that-- where do those prompts end up?
Do those prompts end up back in the ChatGPT model?
And is there a privacy issue associated with that?
TED BENSON: That's a great question.
So the question was-- and I apologize, I just
realized we haven't been repeating all the questions for the YouTube
listeners, so I'm sorry for the folks on YouTube
if you weren't able to hear some of the questions.
The question was, what are the privacy implications of some of these prompts?
If one of the messages is so much depends upon your prompt
and the fine tuning of this prompt, what does that mean with respect to my IP?
Maybe the prompt is my business.
I can't offer you the exact answer, but I
can paint for you what approximately the landscape looks like.
So in all of software, and so too with AI, what we see is there
are the SAS companies, where you're using somebody else's API.
And you're trusting that their terms of service will be upheld.
There's the set of companies in which they provide a model for hosting
on one of the big cloud providers.
And this is a version of the same thing, but I think
with slightly different mechanics.
This tends to be thought of as the enterprise version of software.
And by and large, the industry has moved over the past 20 years
from running my own servers to trusting that Microsoft, or Amazon, or Google
can run servers for me.
And they say it's my private server, even though I know they're running it.
And I'm OK with that.
And you've already started to see that--
Amazon with Hugging Face, Microsoft with OpenAI ,
Google 2 with their own version of Bard, are going to do these,
where you'll have the SAS version, and then you'll also have the private VPC
version.
And then there's a third version that I think we haven't yet seen practically
emerge, but this would be the maximalist,
I want to make sure my IP is maximally safe version of events,
in which you are running your own machines,
you are running your own models.
And then the question is, is the open source
and/or privately available version of the model as good
as the publicly hosted one?
And does that matter to me?
And the answer is, right now, realistically, it
probably matters a lot.
In the fullness of time you can think of any one particular task
you need to achieve as requiring some fixed point of intelligence to achieve.
And so over time, what we'll see is the privately obtainable versions
of these models will cross that threshold.
And with respect to that one task, yeah, sure, use the open source version,
run it on your own machine.
But we'll also see the SAS intelligence get smarter.
It'll probably stay ahead.
And then your question is, well, which one do I care more about?
Do I want like the better aggregate intelligence
or is my task somewhat fixed point and I can just
use the open source available one for which
I know it'll perform well enough because it's crossed the threshold?
SIL HAMILTON: So to answer your question specifically, yes.
You might be glad to know that ChatGPT recently updated their privacy
policy to not use prompts for the training process.
But up until now, everything went back into the bin to be trained on again.
And that's just a fact.
So I think pizza--
it's now pizza time.
Yay, OK.
[APPLAUSE]
[INTERPOSING VOICES]