>> SAM GREEN: Hi, everyone.
Welcome to our seminar.
My name is Sam.
HUGH ZABRISKIE: I'm Hugh.
SAM GREEN: And we're going to talk today about JavaScript and the Web Audio API.
Just to start out, this is an outline of our agenda for the seminar.
We're going to start by talking about why you should be interested in the Web
Audio API, why is JavaScript the language you need for it,
and then talk about JavaScript essentials-- so like,
walk you through some basics of the language,
and then talk about the audio API at a high level.
Then, Hugh will talk about some of the stages of audio production
and then demo this awesome sequencer project he built and show you the code.
And then, we'll have time for questions at the end for people
who are here live.
>> HUGH ZABRISKIE: Cool.
>> SAM GREEN: Cool.
>> HUGH ZABRISKIE: Cool.
I will back up.
>> SAM GREEN: So, first things first.
So one of the great things about the Web Audio API
is that there's no set up required.
It comes built-in to most modern browsers,
including Chrome, Edge, a whole bunch of others-- all the ones
that large portions of people are using today.
So there's no set up, aside from just getting
a web server going, for you to get started working
on your project, which is great.
>> We recommend pretty heavily that you consider
using Chrome for JavaScript web development,
just because its developer tools are really strong.
As an example of just what we mean by saying open up your JavaScript
console-- if you go into Chrome and you look at any web page,
and you left click Inspect Element, and then
you go to this little drop-down right here and you click on Console,
you'll see what opens up looks a lot like a command prompt that you
might see on your Mac, or on the ID.
And just like that, we can type commands here, like Clear,
and other commands like that.
We can create variables, as we'll see later in JavaScript.
And so anything we can do in JavaScript, we can do with the console,
and that's a super handy way to start playing around with APIs
and getting comfortable with JavaScript right off the bat.
No set up required, which is really nice.
Cool.
Awesome.
>> So just one more thing to add.
If you have any questions-- there are many of you who are not here live,
feel free to email us-- these are our email addresses.
If you have questions you don't want to ask us,
like, oh I have a bug in my code, or something
that's a little more specific, maybe Google it first.
There are a lot of great resources about the Web Audio API out there.
It's really well documented and it's being
used by a ton of people in industry, and people who are just
building fun stuff for themselves.
So there should be a lot of resources out there.
Awesome.
>> Cool, so why the Web Audio API?
This diagram is a little bit of an evolution of the way
sound on the web has grown over time.
Bgsound was like the original HTML tag that Internet Explorer used to support.
It only allowed for pretty basic sounds, the functionality wasn't very robust,
and you couldn't do complicated sequencing,
or control when sound started and stopped very robustly.
So, it wasn't particularly well developed.
Then after that, Flash came along-- which,
I'm sure that you guys are all familiar with Flash-- maybe not how it works,
but you've certainly seen it.
You've got to update your Flash Plug-in, all that kind of stuff,
and that certainly extended the range of functionality that was available.
But making the user install a plug-in is definitely
a drawback to including Flash in your application, right?
Because then you're dependent on the user going and finding this plug-in,
and probably being turned off by this extra step
they have to take to use your app.
And then there could be an update that'll break your whole application,
and it ends up being a nightmare for the developer, too.
So that was a barricade.
>> And then after that came along, the HTML audio tag, which
is a feature of more modern HTML-- which certainly allowed for a lot more stuff,
but even the things you could do there were a little bit limited just
as a result of the things that HTML was capable of.
So when the JavaScript API, the Web Audio API,
became a standard practice across browsers,
that really broadened the set of opportunities for developers
to really get into building cool stuff for the web.
For a long time there had been really robust tools
for native audio applications, like-- everyone knows GarageBand,
and then obviously there are more professional audio mixing applications,
and that kind of stuff.
But there wasn't a really good Cloud-- not
Cloud, yeah, I guess Cloud-- web-based platform
that would allow developers to build applications for people
to do audio mixing.
And as he will show you later, the Web Audio API
allows for really powerful stuff to happen really simply,
which is pretty cool.
So that's the instruction to why you should watch the rest of the seminar,
basically.
>> And now, I'm going to talk about some JavaScript-- just basic elements
of the language, so that we can be on the same page
when we talk about the API a little bit later.
Cool.
>> So, this is a summary.
I forgot this was here.
Yeah.
>> HUGH ZABRISKIE: There's two slides here.
>> SAM GREEN: This is the summary of some of the limitations
of the other binding, old methods.
And then now, we have these things.
Cool.
Awesome.
>> So, JavaScript essentials.
First things first, there's a pretty significant difference
in JavaScript versus in a language like C, in the way
that variables are created.
So in C, we're used to having to type our variables, right?
And I don't mean type like type them in, I mean type
like assign them a type-- meaning like, an int, a float, a char.
In C, we were really used to having to create a variable
and then stick to that type for the entire time that we use that variable.
And that isn't necessarily worse, but it's probably harder to use.
One of the cool features of JavaScript is
that variables are what's called "dynamically typed," which
means that I can create a variable with that syntax,
varX equals 5, for example.
That originally creates an integer variable--
right underneath the hood somewhere-- but I
can change that variable to refer to a string
without doing anything like creating a new variable.
I don't need to worry about the type changing.
JavaScript knows that the type's changed, and that happens dynamically.
>> So, there are benefits and drawbacks to that,
as anyone who's worked in JavaScript for a while might know.
There are times when you might accidentally
change the type of a variable and not handle that type changing,
and then your JavaScript can crash-- or an exception
be thrown, because you'll have the wrong type when you expect one type.
Cool.
>> So, scoping-- which is like, if we remember the early weeks in the course,
refers to how visible a variable is and in what area of the code.
All of that looks very similar to the way it looks in C.
So variables are scoped generally within curly braces within a function,
and then there are also globally-scoped variables that
are-- if you write a variable outside of a function,
it will be visible in the entire text.
>> One difference between JavaScript and C in particular,
is that if you declare a global variable anywhere in a text file
it's visible in any function within that text file.
That's correct, right?
>> HUGH ZABRISKIE: Yep.
>> SAM GREEN: So that's also a little bit funky in comparison to C,
where we always had to have our variable definitions above the places
they were used.
That's not a rule that's enforced anymore, so, a little bit different.
And again just to reemphasize, global versus local variables--
very similar to C. You could have two variables with the same name,
and have one of their names be shadowed by a local variable if one of them
was global.
So, similar kind of problems that some of you
may have run into in some of your problem sets so far.
Cool, so that's variables.
>> Control flow, meaning like, if-else-- logical stuff-- and loops.
So to start with, this is what if-else statements look like in JavaScript.
The placement of the various things on the lines isn't important.
This is just one of the conventions for the way we structure code.
Just like in C, we have an "if," a parenthesis statement.
That's not what I meant to do.
I did it again.
>> HUGH ZABRISKIE: Trying to exit?
SAM GREEN: No, I'm just trying to zoom in.
It doesn't matter.
>> So, we have an "if" statement and we have a condition inside of it
that evaluates to true or false, and that determines whether or not
we enter that block of code.
And likewise, we have an else-if, and an else, just like we're used to in C.
>> You also should be pretty comfortable right off the bat with loops,
because they also look a lot like C looks.
But you'll notice again that we have, instead of int initializations,
we have var initializations.
And I guess you have to be careful to make
sure you don't change the value of I from an int to a string,
for example, because that's going to cause weird behavior you might not
expect.
But this should look pretty familiar, as well.
>> So this is where things start to get a little bit crazy in JavaScript
for someone who is going from a background of C. There are functions
in JavaScript, and there's one way to declare a function that looks
sort of similar to C, and then there's another one that
looks kind of different.
>> The first version, which we can see here,
is kind of C-like, where we say, this is a function,
give it a name, give the number of arguments,
and then the contents of the function go inside those curly braces.
We'll see an example of arguments in just a second.
>> Whereas on the next line, we see, oh, here's a variable called "myFunction,"
and we equal it to this generic thing-- function-- that
doesn't seem to have anything going on.
The reason that's different than C is that JavaScript
is what's called a functional language, or has functional elements, which means
that functions are actually values.
And that means that we can set a variable to equal a function
and then move that function around, pass it as an argument,
do all kinds of stuff like that with functions.
>> One other thing to note-- functions are written
with a certain number of arguments.
We'll see an example of a function with an argument on the next slide.
But JavaScript won't yell at you if you try
to use a function with the wrong number of arguments.
It'll just do its best to make do, meaning that if you pass,
you call a function that expects an argument with no argument, all that
will happen is it'll do its best to try and execute that code,
and if it eventually runs into an exception or an error,
it'll throw that exception and just keep going-- which is just one of the ways
that JavaScript works.
Yeah.
>> AUDIENCE: What happens if there's too many arguments?
SAM GREEN: So the question was, what happens
if there are too many arguments?
And the answer is that JavaScript will just
ignore the ones that are after the ones it expects.
It'll try to execute the function call as if it was just the first two.
Right?
>> HUGH ZABRISKIE: That's right, yeah.
Similarly, if there are too few arguments,
it just kind of gives null to all the arguments it doesn't have any values
for.
>> SAM GREEN: Which can actually be handy, if you
want to write a function that takes a variable number arguments.
You can set default values in the definition of the function,
and it can ignore the fact that the input's not there.
So I want to talk a little bit more about this last bullet
point, which is functions are values.
This is an example that is a little bit mind-blowing
if you just read it, and don't think about what's going on for a second.
So, let's look just at the first line here.
We have this variable, f1, that we say is a function that does this thing.
And the contents of the function are console.log('hello').
You can think of console.log as the JavaScript equivalent of printf.
So what will happen is, if we run this code in our browser,
it'll print out a string.
I can demonstrate that.
AUDIENCE: By log, though, does that mean it's being recorded somewhere?
SAM GREEN: Yeah.
So I'll show you what's going to happen.
So the question was, what does log mean?
>> HUGH ZABRISKIE: So console.log is like printf for C.
SAM GREEN: So console.log is like printf,
so if I have this console.log('hello'), and I call that, the string "hello"
gets printed out to the console.
This is the console.
It's just like printf, where it prints to standard out.
And we'll see in a minute, but this is actually
referring to the console object, and calling a method on that object.
That'll make more sense in a minute when we
get to talking about objects in JavaScript,
but I thought I would just mention that.
HUGH ZABRISKIE: We're used to in C, right--
we usually write a big program in main to do anything.
But what's cool in JavaScript is you have this kind of interpreter that
runs in real time, so it takes just line by line,
it can just interpret that on the spot.
And it keeps track of things that have run before,
so it's a pretty useful tool to use console.log, or the console,
generally, for just playing around with JavaScript.
>> SAM GREEN: So going back to this example-- the second line of code
here is pretty mind-boggling in my head.
The first time I read this, it was like, what's going on?
So what's happening is, this function declaration says,
I have a function called f2 that's expecting one argument, f,
and then it calls that function, f, which
was passed to it as an argument with no arguments itself.
So, that might have been confusing.
If we understand this as f2 takes f1 as an argument, and then inside of f2,
f gets called-- which means that this line of code,
after these two lines of code, results in "hello"
being printed to the console.
>> The fact that we can pass functions around as values
ends up being one of the most powerful features of JavaScript
as a programming language.
Outside of all of the awesome things it can do,
just as a feature of the language in terms of the way
that it makes things easy to program and allows
for things that aren't particularly well-suited to the web,
functional programming and functional programming aspects of JavaScript
is one of the most powerful concepts that
exists in JavaScript-- if you ask me.
Cool.
>> So, next thing.
In addition to being functional, there are also elements of JavaScript
that are object-oriented, which is one of the very
popular buzz words in computer science.
Object-oriented programming is a really popular thing.
JavaScript has a version of that, where I believe every value is also
an object, which means that every object wraps together some number of values.
So for values that are simple, like an integer, like varX equals 5,
that object just wraps that one value.
>> But we can also imagine a situation where-- we can think of situations in C
where we wanted to do something with structs,
for example, that wraps several values together and makes
it really easy to pass things around.
That's when an object is in JavaScript.
>> It's important to remember when I say that objects wrapped
some number of values together, that functions are also
values, which means that functions can also be inside of a JavaScript object.
And the reason that's important is that, whereas we often
think of calling a method on an object that's
of a popular term from other popular object-oriented languages,
one of the differences here is that all that a method is in JavaScript
is a value stored inside of an object that performs some action-- possibly
using the other values that are inside of that object, but not necessarily.
So you can imagine a situation, I guess in a little bit of a crazy way,
where you called a method of one object on another object, for example.
So, it's a little bit funky in that way.
>> And you can also change the methods that are associated with an object
by assigning that method a new function, which is also
pretty different from other object-oriented languages, where
once we declare an object and instantiate it,
we can't change the methods that are associated with that object anymore.
So that's pretty different.
Cool.
>> So here's an example, first, of an object in action.
This is what's called a generic object, which
means that it doesn't have any particular name, doesn't have a class,
it's just some wrapping of values.
And the way that looks is, we have this outer pair of curly braces here
that indicate to JavaScript and say, this is an object.
The values inside of it are each values inside
of the object that should be wrapped together.
And inside of that object, we then have key value pairs,
where the key refers to the name of the value inside of the object,
and the other side-- opposite the colon here--
is the actual value that should be stored.
>> So you see here that we have a key called fn with value sam,
followed by a comma, saying onto the next entry.
Then a key called ln, with a value of green,
followed by a comma, followed by "print,"
which is going to have a function value that is going to do this line of code.
Let's take a step back and unpack what's going on here.
So this is a little bit complicated, and we're seeing something new
for the first time.
The "this" keyword is the new thing we're seeing here, and what this does
is, refers to the current object in scope, right?
So when we say, this points all the way back
to this entire object-- when we do this.fn,
we're going to go all the way back to this object, go to the fn value
and get sam, pull it all the way back, stick it here, and then move on.
>> AUDIENCE: So with the retrieval, is that done because of the parameter
definition?
SAM GREEN: So the question was, is the retrieval done because of the parameter
definition?
Yeah, absolutely.
What's going to happen here is, this dot says to the JavaScript,
OK, I'm getting some value from this object from myself.
And then it'll look for an entry called fn, and if it finds it,
it'll return that value-- so, it's sam.
But I could also have typed something that was not defined here,
and then it would just return undefined-- which
is a thing that JavaScript can do, which can have benefits,
but it's also-- if you make a typo, it can result in weird errors.
So it'll just try to find whatever you tell it to find
and it's not going to complain if doesn't find it.
It'll just say, I didn't find it, and then move on.
So it would be undefined, plus blank, plus last name.
Yeah.
And then we can see that if we could then go down and access--
and we call tf.print() with parentheses.
It's going to call that print function with no arguments, right?
But if we just said tf.print() semicolon, without the parentheses,
all that would have done is pull out the function from the value,
but not actually called it.
Cool.
HUGH ZABRISKIE: Should we make an object?
SAM GREEN: Sure, let's do that.
So I can move this example to the console.
We can imagine that I have an object.
So this is a simple object.
This is an object that contains two values with two keys, two key value
pairs.
So I can then access the value stored inside of this object by doing x.x1,
for example, and I get 1 back.
Likewise, x.x2, get that value back.
>> And now the really cool thing is, I can actually add something to this object
after I've created it.
So you can imagine, let's say I have a function.
>> HUGH ZABRISKIE: You have to do Shift-Enter.
>> SAM GREEN: Oh, that's annoying.
What did it not like?
Oh.
Here we go.
Cool.
>> So I've just created this function, f, that
is going to go to the current object and print this.x1.
So if I just call f by itself, nothing's going
to happen, right, because there is no x1 field in the object it's referring to.
But, if I say, x.f = f, and then I call x.f(), I'm going to get back 1.
That f function is now associated with the object x,
which has a key called x1 associated with the value 1,
so when we call this.x1, it's going to find what it's looking for
and be able to print a value out.
So that's just one example of kind of the crazy things
you can do with objects in JavaScript.
>> So that version was the generic version, meaning
that we've created an object using this parentheses notation-- brace notation,
rather-- and that's handy if we just want
one instance of a particular object, but what if we want to have more than one
of the same kind?
And the answer to that question is, there are things
called classes in JavaScript as well.
We can create a function that does some sort of initialization
for a foreign object, and we'd say, like,
my class-- so the name of the reusable object--
equals function that sets it up.
So what this would be equivalent to is creating an object that
would be just like, curly brace, str, colon,
this is a string, semicolon, curly brace.
That would be the generic object we initialize,
with the one difference being on the next lines we create a prototype, which
means it's a default key that we add to our object that
has the value listed here.
Meaning that, when I create a new instance of this MyClass object,
it's going to have pre-built inside of it a value called str and another value
called myPrint, which is going to be a function.
Awesome.
>> Great.
So the last thing to say about JavaScript
is that it's really useful for what are called asynchronous operations.
Asynchronous means is that we can wait for some operation
to complete before we move on, but move on while we wait
and then have something happen later on.
And what I mean by that is, you can imagine a situation where
you send a request to some web server somewhere,
and it's going to to send you back some big chunk of data, right?
And your user could wait in the meantime for that to happen,
and nothing could be going on at that time.
But that's not a great design, right?
You don't want the web page to freeze.
What if the user wants to click on a drop-down menu?
It's not a great design pattern.
Instead, basically what JavaScript does is says,
OK, do this operation asynchronously.
So like, wait in the background, and then when the operation is done,
call the callback function-- call some function,
do some action-- to signal that the operation we were waiting for to end
is over.
And the reason that's super powerful is, we can do something, pass an argument,
do something, and then wait for something to happen.
Then, once that something completes, we can call a callback.
That's really handy because it lets us do things with Web Audio API,
for example, like load an audio file from a remote server
without having to wait for the entire audio file to be loaded,
which would be really bad for user experience.
Cool.
>> Last couple notes about debugging, since this
is a thing you're going to have to do as part of your project, guaranteed.
I mentioned the JavaScript console.
It's a super useful feature of all modern browsers,
And we really encourage you to get comfortable using your console,
if you want to get good at JavaScript.
It's super handy for debugging, but it's also
really useful for figuring out how to use an API.
It allows for really easy experimentation
without having to type some code, and then compile it.
You don't have to do all those steps.
You can just write some code into a line,
and then get immediate feedback on whether or not that line of code
worked-- very handy.
>> And also, just one technical note-- the JavaScript console is an example
of a REPL-- so that's R-E-P-L, REPL, which stands for read, evaluate,
print loop.
You're going to type some stuff in, it'll read what you typed in,
it'll evaluate it, and it'll print the output, and then it'll start again.
That allows you to rapidly go in circles iterating, which is really cool.
>> I guess real last note-- this is the actual last note, yes.
How do we actually use JavaScript?
So first, we can import it using a script tag
at the top or bottom of an HTML file-- anywhere inside of an HTML file,
really.
And within a script tag, there are two sub-ways of importing JavaScript.
The first is by having a separate JavaScript file
that we import in its entirety, or by having an area of code like script
to start, and then backslash script to end.
And then we just write JavaScript inside the HTML file.
Those are the two ways.
You can't have it inside of HTML.
AUDIENCE: Is one better than the other?
SAM GREEN: The question was, is one better than the other.
So, yes, as a coding style practice, and also it's like a design practice.
There are two reasons why it might be better.
The first is, it makes your code a lot more readable if all of your HTML
is in one place, all of your CSS is in another place, all of your JavaScript
is in a third place.
Right?
I think we should have already talked about it in sections-- like CSS-- what
that is-- and it goes often in another file.
So, similar kind of concept here.
You can also imagine that JavaScript would be reused on more than one
HTML page, or perhaps a great many HTML pages,
and having that JavaScript refactored into one
file that you can import into more than one place
allows the code to be way more maintainable.
You can imagine making one change to the JavaScript
and having to change it in 100 different files.
And instead we can just change it in one, which is way more powerful.
Did I answer your question?
Cool.
>> We can also type into the console, as we've mentioned before.
And again, one last note-- Web Audio is built in,
you don't need to load anything.
Cool.
Are there any questions, do you have any more questions about JavaScript,
before we move on?
>> AUDIENCE: [INAUDIBLE]
SAM GREEN: All right, cool.
So now he's going to talk about the API.
>> HUGH ZABRISKIE: Cool.
Thanks, Sam.
>> SAM GREEN: Sure.
HUGH ZABRISKIE: Awesome, so we'll move on from JavaScript.
So we've talked about some of the essentials of JavaScript,
and those are the variables, functions, objects, functions as variables,
asynchronous loading.
These are all things that you'll see as you use the Web Audio.
So we're just going to talk about it first at a high level.
>> It's an API, so it's something that's built, as Sam said,
right into the JavaScript that you use in the console.
And it's actually just like C++ code that is really built into Chrome
and Firefox, and all of these browsers.
So the main idea with Web Audio is that you have
this kind of pipeline of audio, right?
So your audio data comes in in some form.
>> There are kind of three main forms-- you have the oscillator, which
creates a sine wave, cosine wave, we're going to see how that works.
Another very common one, of course, is an MP3.
So maybe you start with a song, and then you
want to do some filtering to that and output
that-- that could be a possible source.
And then a really cool one is the microphone.
So you can use some very basic calls in JavaScript
to get access to the microphone, and so if you
wanted to make an app like a pitch detector,
for example, that takes in your voice and figures out
the pitch-- very easy way to that.
You can just kind of read it in, figure out the frequency,
and then output a number.
So we'll see how that works, as well.
>> The destination is basically where the audio data is output.
So generally, that's like your laptop speakers.
Other options are like a ScriptProcessorNode--
we'll get to nodes in a second-- but basically,
either you're putting sound out through your computer through speakers,
or you're kind of recording it, so you're storing it as audio data.
So maybe if someone creates music in your app and then
you want to record that and maybe like export it to SoundCloud, for example--
that would be one way to do that.
All the fun stuff, which we'll talk about,
happens between these two points, where we load in the music
and then output it.
>> So I'm going to talk about the five stages of audio production in a second.
We have this thing called an AudioContext, which
is this little wrapper we see here.
Basically what AudioContext is-- if we go to the JavaScript console right now,
we can create one right now.
Just an example of REPL, right?
We're reading, evaluating, and it prints.
>> AudioContext is a global state.
It's a struct, it's an object here, and it keeps information
about things that are going on on the screen related to audio.
One example is the current time.
This tells you the number of seconds, very precisely,
since the web page loaded.
So this is a really useful little property that you can use.
It's read only-- I think actually you can try to set it a value.
It'll tell you set it, and then if you print it
again-- it didn't actually quite work.
So there are read-only properties in JavaScript.
This is really useful if you're kind of syncing
a lot of different information, when you're
kind of playing different sounds.
>> Another really useful one is the context destination.
Definitely, if you're interested, be trying this at your own console right
now.
So this is an AudioDestinationNode.
Basically what this says is, where is the output going?
So there are two real options here.
Usually the default is just your speakers,
so AudioDestinationNode basically just says
there are zero outputs to the sound coming in, sent to the speaker.
So generally, you don't have to play with that.
If you're interested in actually using the ScriptProcessorNode for recording,
definitely shoot me an email later because that's
a little more complicated.
But generally, you're just kind of outputting sound in some form.
So cool, we'll jump back here.
AUDIENCE: I'm sorry.
HUGH ZABRISKIE: Yeah.
AUDIENCE: I know you said to talk to you later about recording.
Can you interface that with Pro Tools?
>> HUGH ZABRISKIE: With Pro Tools?
Let's see.
I don't think so.
So going between the client, which is the JavaScript
console, and your actual computer, is generally
something that's kind of off limits, if you
will, kind by the nature of the-- it's kind of a design thing,
but you try to keep the browser separate from the user's actual computer.
Generally, the only thing you're able to access is the microphone or the camera.
You're not able to, I don't think, use Pro Tools.
However, if you created a track in Pro Tools,
exported that, you could load that in here, filter it, for example,
process that, and record that into an Audio Destination-- or, no-- a Sphere
Processor Node.
And then from there, you could export that to SoundCloud, you
could send it in an email, or whatever you like from there.
>> But there is kind of a slight barrier between making music on your computer
and making music online.
>> SAM GREEN: And that's not unique to this API.
It's a security feature of Chrome, and I think every other modern browser.
The browser is self-contained.
So for example, a web page can't use JavaScript to turn the sound
on on your speakers, for example.
Or it can't turn your computer off.
And there is no intermediate point between those two things, right,
so either you have a complete abstraction,
or you open up the security flaw of letting
a programmer with bad intentions do whatever they want with your laptop.
And that's why Chrome is self-contained.
>> HUGH ZABRISKIE: Yeah.
Does that make sense?
Cool, cool.
I was just going to show an example of one.
This is pretty much as far as you get, in terms
of accessing the user's computer.
If you have a USB keyboard plugged in, you can use something called the Web
MIDI API, which we won't really talk about here,
but this is another API that's built into at least Chrome-- again,
this is why we love Chrome-- I think Firefox or Safari,
this is an easy thing to google-- different browsers have
different support for which APIs they have implemented.
But if you wanted to plug in a keyboard and work with that information,
kind of send the keyboard information over to the computer
and then use that online, this API is where you'd be working that.
>> Cool.
OK.
So, quickly moving on here.
How are we doing on time?
>> SPEAKER 1: About 15.
>> HUGH ZABRISKIE: 15 minutes left?
OK, cool.
So we'll race ahead here.
>> So basically, the main point of thinking of this as a pipeline
is that each step in the pipeline is a series of audio nodes.
Our source, let's say, is an oscillator.
We need to create an oscillator node.
And that is just kind of the little function--
and they're all based out of the audio context here.
>> AUDIENCE: When it said oscillator, does that mean
it's actually literally going from two different poles back and forth?
>> HUGH ZABRISKIE: No, it's like a digital representation.
It's actually implemented in C++.
I actually don't know the specs of how it's actually implemented,
but all this is working as binary data.
Actually, yeah.
That would be saying, I could actually, if you're interested,
I could send you a little more information about how waveforms
are kept having a digital format.
OK, cool.
>> So we're generating a tone like a sine wave or something like that, maybe
440 Hertz.
We create an oscillator.
If we want to set the volume, we connect anything to a GainNode,
which we could do with .creategain.
That sets your volume.
You can pass that onto any of the other options-- well,
so an audio buffer source node is where you might
store an MP3 that you've loaded in.
>> Biquad filter is for filtering if you want to take all the base out
of a song, or something like that.
God forbid you want to take the base out of a song.
And AudioDestination node is, again, like where our finalization is.
If you're ever interested in seeing all the different possible options,
just go to the tab and let the auto-complete come up.
And if you do create, you'll see all the different things that you can create.
You can create dynamic script processors,
I don't even know what that is, for mixing channel mergers
and channel splitters and all that.
Cool.
>> So this is just an example of a pipeline.
So we have three sources coming in.
Maybe these are waveforms, maybe these are MP3s.
One's going through a filter, another one's
getting distorted another one's panning left and right.
You can do all sorts of things and they all get mixed around together,
and then out comes the audio at the end, as destination.
This is an example of what more complicated Web Audio code looks like.
You're creating all these different objects right here--
I'm not sure of this.
No, it doesn't zoom in.
OK.
>> SAM GREEN: You do Control, Scroll-Up.
HUGH ZABRISKIE: Control Scroll--
SAM GREEN: No, no.
Control--
>> HUGH ZABRISKIE: Oh, Control, Scroll?
Oh, gotcha.
Yeah.
Wow, nope, nope.
OK.
I will not do that.
>> So yeah, in this first section here, you see
we're creating all these different nodes out of the context.
We're just piecing them together in the second part
by this function called Connect.
That's a really key function in Web Audio.
It just means once you've done something with the sound in one node,
pass it on to the next node.
So we have the source, it connects to the analyzer,
the analyzer does something with it, it goes to distortion, and so on,
and to the destination at the bottom right here.
Cool.
OK, so we'll keep moving on.
>> The pipeline-- again, these are the most common pipelines,
so we talk about all these things like distortion, panning, all this stuff.
If you're really interested in using things Pro Tools,
those probably interest you.
If not, maybe you just want to play the sound,
or maybe you just want to set the volume on the sound.
Those are the two most common sort of pipelines in audio production.
>> Again, the ways you can take it in as an oscillator-- so, let's
do a the demo of that right here.
So we're going to create a simple audio context here,
and from that we're going to create our oscillator.
So that is, again, we're just going to call Create Oscillator.
We're going to set a frequency on that, 440 Hertz, everyone's favorite.
Then we connect that to the destination point-- which is the speaker, so
the context destination.
Finally, we just say, start zero seconds from now, and do we have sound?
>> [RINGING]
>> HUGH ZABRISKIE: Here we go.
It's just a sine wave.
OK, cool.
And then we'll stop that.
>> AUDIENCE: Where did that feedback come from?
>> HUGH ZABRISKIE: The feedback?
Oh, probably our microphones.
So yeah, that's how you do it.
And actually, if I had kept it running, you
could have the frequency value as it's running,
so that's a fun thing to play around.
Cool.
That's always a lovely one to present.
>> SAM GREEN: We didn't think about that, did we?
HUGH ZABRISKIE: Yeah, that's a nasty one.
So, buffer loading-- I'll show an example of that at the very end.
That's loading an MP3.
And microphone, you use just a function called Navigator.getUserMedia()
to request access to the user's microphone for that information.
>> Here's filtering, I'll just keep moving from this.
This is pretty high level, but filters just allow you to
>> [BEEPING]
Filtering also allows you to create things like pink
noise, brown noise, white noise.
If you want to create pure noise, which some people love to mess around with,
you can use Web Audio filtering to do that.
>> Audio Panning-- so imagine if you're writing a game
and you want the sound to sound like it's coming, like,
shooting across the screen, you can use the panning of the audio
to create this kind of cone, which like-- it's pretty mathy,
but it's actually really cool if you get it work,
and there's some good tutorials on it I can send you.
Basically, you can kind of create the sound
of something going by in a 3D way.
And if you have a DJ interest, you can start mixing and cross fading songs.
>> This is just some very basic code, basically what I did before.
This sets the volume of the oscillator, so we create our oscillator
which creates the waveform.
We create our GainNode, set our frequency,
and then connect the oscillator to the GainNode, which then basically changes
how much signal is allowed through.
But really, it's a digital thing, so it's more just-- yeah.
That's not what's actually happening, but that's what happens in real life
with a gain.
>> AUDIENCE: --quantization of the volume parameter?
HUGH ZABRISKIE: Sorry?
AUDIENCE: Is it a quantized volume parameter?
HUGH ZABRISKIE: Yeah.
And this is one thing I'm really deficient on in my knowledge,
how gain works on a digital level.
I know with actual signals, it's basically
controlling how much you're amplifying the signal.
So, yeah.
I'll send you more information about that, because I'd be curious actually
to know more about that.
But basically the parameters are, one is the fold--
the louder signal-- and zero is no signal, or you won't hear any sound.
We'll skip demo time for that because it's basically what I did before.
And again, the Context.Destination is the audio destination node.
Awesome, OK.
>> So I'm going to do a quick two demos.
How are we doing on time?
>> SPEAKER 1: About 10 minutes.
>> HUGH ZABRISKIE: 10 minutes?
Great!
Awesome.
>> So the first one I'm going to do, it's called My Favorite Song.
So this is just a little HTML JavaScript.
We're going to have two buttons on the page play my favorite song
and stop my favorite song.
I'll change this.
>> AUDIENCE: Cover your microphone.
>> HUGH ZABRISKIE: Yeah.
And I've loaded in here a script that basically--
and this is really useful for loading an MP3,
so this just makes loading MP3s way faster.
It's basically just a wrapper.
It just makes the process of loading in MP3s much faster,
otherwise you're using HTTP request, kind of like what we were doing
on the current piece set with Server.
It's really ugly, you don't want to do it.
>> So this guy, Boris Smus, wrote a really useful little tool called BufferLoader.
All you do is simply pass it the context, you pass it a list--
or, yeah, is it a list in JavaScript?
SAM GREEN: An array.
HUGH ZABRISKIE: Oh, it is an array, that's right.
It's an array of paths to different files.
And then you pass it a function.
This is the callback we were talking about with asynchronous loading.
That will be called once the files loaded.
And that function that is called when the file's loaded takes as a perimeter
an array of loaded buffers.
So that occurs here.
Basically, BufferList is going to be one value--
or it's going to be an array of length one, that has in it in index
zero the entire loaded file of the MP3.
So what I do when I finish loading is, I simply
create a buffer source, which is an audio buffer source node.
The next step is I load in the source.buffer as the full loaded buffer
from the BufferList-- it's a lot of buffers--
and then you connect that audio buffer to the destination.
So what it's going to do is just simply put the MP3
straight through to the output, and start it immediately
upon getting this call.
>> Cool, so let's see this happen in action.
My [INAUDIBLE] here, let's see.
So I'm just going to start a basic server.
That's something that you need to do if you're
making requests for loading files.
I'm going to start a basic server.
This is basically your entire PSET right now in one line,
but it's just starting a server on port 80/80.
So we go over here, we going to load 80/80,
we're going to go to My Favorite Song.
So if I hit "Play my favorite song" right now,
it's going to load my favorite song and play it--
[MUSIC - THE EAGLES, "LIFE IN THE FAST LANE"]
--which happens to be "Life in the Fast Lane" by The Eagles.
Now, I could hit "Stop my favorite song" and replay it.
>> [MUSIC - THE EAGLES, "LIFE IN THE FAST LANE"]
>> And if I go over to console, because I used a global variable over here
to keep track of this value, it actually will now be recognized in the console.
So it auto-creates for me.
So that's what's playing right now, and I can simply call source.stop()
on that.
Well, you know what?
Just so you guys have heard this song-- you might recognize this song.
>> [MUSIC - RICK ASTLEY, "NEVER GONNA GIVE YOU UP"]
[MUSIC - THE EAGLES, "LIFE IN THE FAST LANE"]
We've now all been Rickrolled.
OK, great, moving on.
Cool.
So this is basically an example of just how you could load an MP3 file--
[MUSIC - THE EAGLES, "LIFE IN THE FAST LANE"]
--and play it, and stop and start it.
I could have done a lot more [INAUDIBLE]
>> The last one I'll do is, I'll show you a [INAUDIBLE].
>> [MUSIC PLAYING]
>> It's like, ogg.wave.mp3.
I think, if I remember correctly, I've run into some issues with .m4a,
but I'm not sure about that.
I think mp3.wave--
>> [MUSIC - RICK ASTLEY, "NEVER GONNA GIVE YOU UP"]
>> OK, great.
I shouldn't have said that.
Anyway, hello.
So we have this open.
So now all I do is, I basically created a basic syntax for creating music.
So if I do something like, add g4 on 1 2, what that means is that,
add the piano note, G4, which is the fourth G
up on the piano from the bottom.
So this is kind of MIDI speak, so for those who are music based,
this is just MIDI notes.
>> AUDIENCE: That's the G of the Middle C, right?
>> HUGH ZABRISKIE: This is the G above Middle C, that's right.
>> AUDIENCE: Above Middle C.
>> HUGH ZABRISKIE: Yeah.
Actually, yes.
I think I actually made it one [INAUDIBLE],
so this might be an octave above that.
So let's see.
If I hit Play--
[REPETITIVE PIANO NOTE]
--we're going to hear that.
The idea is that it operates just like a command line would,
so if I go up and down on my keyboard, you
can go back to previous commands, which is pretty useful.
And below is my list of tracks, which are all running on loop.
>> AUDIENCE: You were assuming the 88-key keyboard on that, right?
>> HUGH ZABRISKIE: The question was, am I assuming an 88-key keyboard,
and yes, I am.
What I did is I basically took 88 samples
of the piano, one for each note.
And so every time you hear a note from now on,
that is actually a loop that looks like-- this is getting played on loop,
so for every note, this is running.
What happens is, I create a buffer again,
I create a gain node to set the volume.
This just a really complicated way of saying I
store the buffer in a source.buffer.
I give it the gain, I connect it to the gain,
the gain is connected to the output, and then I play it.
So that is kind of the process of taking in a buffer source.
>> AUDIENCE: Can you actually take that dry sound and make it wet [INAUDIBLE]?
>> HUGH ZABRISKIE: You can, yeah.
There's re-verb, there's delay, distortion.
You can basically put anything in between in that sandwich of-- well,
pipeline is a better metaphor, but you can add anything in that.
Cool.
>> So I'll finish the demo here to give you a sense
of just the sheer number of times you can run that function all at once.
So I'm going to remove this.
I'm going to create a generator that-- basically what does-- this is really
kind of a complicated syntax-- but it's going to generate notes on the fly,
and just start playing them as it evaluates them.
[INTERPOSING PIANO]
>> So we can just make a little music here.
[INTERPOSING PIANO]
>> So what this command does, for example, is
it takes those three notes for the piano and then puts them on B3.
This syntax might make a little more sense
to those who have a music background here.
>> I can add a kick drum.
I can--
>> [INTERPOSING INSTRUMENTS]
>> --just play around with that.
>> So you can make--
[INTERPOSING INSTRUMENTS]
That one's a little more annoying.
[INTERPOSING INSTRUMENTS]
>> So that randomly adds a dry cymbal on every 16th note, with a 16%
[INAUDIBLE].
>> [INTERPOSING INSTRUMENTS]
>> Yeah, so the way this works-- it's always in 4:4.
[INTERPOSING INSTRUMENTS]
>> Yeah, so the four quarters, and 16/8.
[INTERPOSING INSTRUMENTS]
>> So on average, you get 60% of hits on the 16th notes.
>> Anyways, this was just kind of to show off
some of the things you could build with the Web Audio API.
It's really powerful, it's really fast, and you can make a lot of cool things
with it.
So again, any questions you have, email myself-- Hugh-- or Sam,
and honestly, Google has a ton of good resources.
Any last questions?
Yeah.
>> AUDIENCE: So you can access the built-in microphone.
What if you wanted to use a better microphone?
HUGH ZABRISKIE: If you wanted to use better microphone?
So again, this is part of the abstraction between Chrome
and the rest of your computer.
Unless it's available through an API, like Web MIDI API,
you could probably find some hacks, but generally not as feasible.
>> SAM GREEN: You can also-- all the Chrome knows
is what your default microphone is, and it accesses that.
So if you had a microphone you could set as computer's default microphone,
you could access it that way and it would probably work.
HUGH ZABRISKIE: That's a good point.
I've never tried that, but you might be able to kind
of-- if you redirect the input speaker, you might be able to do that, yeah.
>> Any last questions?
Cool.
Well thank you guys so much for watching.
I'm Hugh.
SAM GREEN: I'm Sam.
HUGH ZABRISKIE: And this is CS50.