00:00:10.400
hi
00:00:11.440
in this speech i'm going to talk about
00:00:13.200
async ruby
00:00:14.880
async ruby is an awesome addition to the
00:00:17.119
ruby language
00:00:18.640
it's been available for some time now
00:00:20.400
but relatively few people knew about it
00:00:22.960
and it has stayed off of ruby mainstream
00:00:26.400
the goal of this talk is to show you at
00:00:28.880
a high level what async ruby is about
00:00:32.559
whether you're a beginner or an advanced
00:00:34.719
rubist i hope to show you something you
00:00:37.360
didn't know about ruby
00:00:39.600
we're going to go through a couple
00:00:41.120
simple examples that show all the power
00:00:44.320
of asynchronous programming
00:00:47.280
we'll also explain the core concepts of
00:00:50.160
how it all works
00:00:52.160
i've been a ruby programmer for 10 years
00:00:54.160
now
00:00:55.039
and this is in my opinion by far the
00:00:58.160
most exciting addition to the ruby
00:01:00.160
language during this time
00:01:05.519
my name is bruno sutic i'm an async ruby
00:01:08.720
early adopter and i made a couple small
00:01:11.280
contributions to it
00:01:12.880
you can find me on github as bruno
00:01:14.799
hyphen
00:01:15.840
i don't do social networks but you can
00:01:17.520
find my contact info on my webpage
00:01:19.759
brunoceutic.com
00:01:23.840
now before jumping into async ruby
00:01:27.200
let's explore what does async really
00:01:29.600
mean
00:01:30.479
what is asynchronous programming
00:01:33.439
i think it's commonly accepted think
00:01:35.119
that javascript brought async
00:01:37.360
programming to the mainstream developers
00:01:39.360
consciousness
00:01:40.880
so i think it'd be fitting to explain
00:01:42.880
asynchronous programming with a simple
00:01:45.520
javascript example
00:01:47.600
also i assume a lot of you have written
00:01:49.600
at least a little javascript because
00:01:51.600
it's so unavoidable these days
00:01:54.720
now let's look at this example
00:01:59.840
we're making a simple http get request
00:02:02.560
to
00:02:03.719
httpbin.org
00:02:05.520
we're registering a promise that runs
00:02:07.920
when the request response is received
00:02:10.959
this function just prints the response
00:02:12.480
status
00:02:13.840
on the last fourth line of this example
00:02:16.720
we're printing some string
00:02:19.120
the output shown below the code is
00:02:21.599
expected
00:02:22.800
this is a simple example of an async
00:02:25.040
program where we're usually making some
00:02:27.680
io request and then something happens
00:02:30.080
later in a callback when the request is
00:02:32.560
done
00:02:34.080
one thing to note in the output here is
00:02:36.720
the program first runs the code on the
00:02:38.800
last line it just prints the string
00:02:41.680
later when the request is done it prints
00:02:44.000
the response status
00:02:46.239
if you think about it it's unusual for
00:02:48.560
simple programs to run backwards like
00:02:50.959
line 1 line 4 then back to line 2.
00:02:55.200
to us developers and humans programs
00:02:58.319
that run top to bottom are easier to
00:03:00.879
understand
00:03:02.879
the point i'm trying to make here is
00:03:05.120
async programs are harder to follow and
00:03:07.760
understand
00:03:09.200
programs that run top to bottom
00:03:11.760
synchronous programs are easier to
00:03:13.920
reason about
00:03:15.599
in the case of javascript as the program
00:03:18.000
becomes more complex
00:03:19.920
they may end up in an infamous state
00:03:21.920
called a callback help or promise hell
00:03:24.560
or even asyncovate help
00:03:28.640
so then why would we want to make our
00:03:30.879
programs asynchronous
00:03:32.640
why not just stick to linear
00:03:34.560
top-to-bottom approach the answer is
00:03:36.879
simple
00:03:37.840
performance
00:03:39.440
to understand this let's look at the
00:03:41.040
following example with some javascript
00:03:42.799
pseudocode
00:03:46.000
here we're making three http get
00:03:48.480
requests and each one takes two seconds
00:03:51.040
to run
00:03:52.400
how long will this whole program run
00:03:54.799
surprise surprise the program will run
00:03:57.519
for two seconds total
00:03:59.680
in this example we are firing three http
00:04:02.000
requests at practically the same time
00:04:04.720
the trick is that waiting for responses
00:04:06.799
happens in parallel
00:04:08.879
asynchronous programming enables this
00:04:11.120
parallel weighting and that's how we
00:04:13.120
achieve these big performance gains
00:04:18.160
if we look at the equivalent code in
00:04:19.680
ruby we'll see that the same example
00:04:22.000
takes 3 times longer to run
00:04:24.479
in this case
00:04:25.680
3 times 2 seconds equals the predictable
00:04:28.880
6 seconds
00:04:30.560
reason for this is there's no parallel
00:04:32.960
weighting on responses
00:04:35.120
ruby is synchronous
00:04:39.360
so
00:04:40.080
how do you make 3 or 5
00:04:42.479
or 100 requests in ruby more performant
00:04:45.680
you use threads
00:04:47.360
in this example we see how to speed up
00:04:49.600
our program with 3 requests in ruby
00:04:52.479
the whole program finishes in 2 seconds
00:04:55.600
this works
00:04:56.720
and now you may be wondering
00:04:58.960
ruby isn't asynchronous by design but it
00:05:01.759
has threads
00:05:02.960
are we good then
00:05:05.919
if you've done any real world programs
00:05:08.240
withdraw ruby threads you know the
00:05:10.560
answer to that question
00:05:12.479
threads in ruby are hard
00:05:15.199
specifically there are two problems with
00:05:17.199
them
00:05:19.039
the first is language level race
00:05:20.880
conditions
00:05:22.240
these are particularly nasty and hard to
00:05:24.479
debug
00:05:25.759
this type of problem can occur with even
00:05:28.160
the simplest threat programs
00:05:31.120
the second problem with threads is
00:05:32.880
maximum number of threads
00:05:34.960
this matters when you want to make a
00:05:36.880
large number of parallel requests
00:05:40.320
i just tried maxing out the number of
00:05:42.160
threads in my machine which is a
00:05:43.919
mid-range macbook
00:05:46.000
the maximum number of threads i could
00:05:47.759
spawn was 2 000.
00:05:50.560
that may seem like a lot but if you have
00:05:52.720
say a million http requests to make that
00:05:56.080
number of threads is not sufficient
00:05:59.360
let's talk about async rubina
00:06:02.080
ac ruby is a new type of concurrency in
00:06:04.479
ruby
00:06:05.680
if you ever think i want to do multiple
00:06:07.840
things at the same time in ruby async
00:06:10.560
may be a good fit
00:06:12.319
for example you want to serve more
00:06:14.400
requests per second with the same
00:06:16.080
hardware
00:06:17.440
you want to make more requests with your
00:06:19.280
api client at the same time
00:06:22.160
you want to handle more websocket
00:06:24.000
connections concurrently
00:06:26.960
ruby has a couple options when you want
00:06:29.039
to make multiple things at the same time
00:06:33.600
first is you can do more work with
00:06:35.520
multiple ruby processes
00:06:37.440
second approach is reactors a new ruby
00:06:40.400
3.0 feature which it seems is not
00:06:43.120
production ready yet
00:06:45.360
third approach which we already
00:06:46.960
mentioned them are
00:06:48.400
threads and lastly we now have async
00:06:54.560
so
00:06:55.280
what is asyncruby how do you run it
00:06:58.639
async is just a gem and you install it
00:07:00.960
with
00:07:01.680
gem install async that's it
00:07:04.400
it's a very very nice gem because matt's
00:07:06.720
invited it to ruby's standard library
00:07:09.360
the invite has not yet been accepted
00:07:12.080
gem creator is samuel williams a ruby
00:07:14.639
core committer so you can kind of feel
00:07:17.599
ruby core team including max himself are
00:07:20.160
backing this gem
00:07:23.759
async ruby is also an ecosystem of gems
00:07:26.720
there's async http a really powerful
00:07:29.520
http client
00:07:31.039
there's async of 8 gem which is just a
00:07:33.440
synthetic sugar
00:07:34.960
falcon is a highly scalable async http
00:07:38.000
server built around async core
00:07:40.800
we have asyncretis async web sockets and
00:07:43.599
others
00:07:44.800
this talk will focus the most on the
00:07:46.720
core async gem and the accompanying ruby
00:07:49.280
language integration
00:07:51.360
let's do an async ruby example that is
00:07:53.599
equivalent to javascript example we had
00:07:56.000
before
00:07:58.639
in this example we're using async http
00:08:01.280
gem
00:08:03.199
the only thing you have to know about it
00:08:05.039
is it's an http client
00:08:07.759
you call get on it and it makes the
00:08:09.840
request
00:08:11.199
the actual code starts with a
00:08:12.720
capitalized async a current method with
00:08:14.960
a block
00:08:16.960
all the asynchronous code in a ruby
00:08:19.039
program is always wrapped in this async
00:08:21.599
block
00:08:22.960
async ruby has a concept of tasks
00:08:27.680
we spin multiple tasks when we want to
00:08:29.840
make things run concurrently
00:08:32.399
in this example we're running three
00:08:34.240
requests at the same time
00:08:36.560
and just like in the previous javascript
00:08:38.240
example all three requests are started
00:08:41.039
at virtually the same time
00:08:44.000
the big win is that waiting on the
00:08:45.839
responses happens in parallel
00:08:54.560
the total running time of this program
00:08:56.399
is slightly more than two seconds it's
00:08:58.560
not exactly two seconds because of the
00:09:00.480
network latency
00:09:02.880
this basic example shows the general
00:09:05.279
structure of async ruby programs
00:09:07.760
you start with an async block that is
00:09:09.920
past the main task
00:09:12.000
that main task is usually used to spawn
00:09:14.399
more async subtasks
00:09:16.640
these subtasks run concurrently to each
00:09:19.040
other and to the main task
00:09:25.680
just to make it explicitly clear
00:09:27.839
async tasks can be nested indefinitely
00:09:31.120
so a task block is past a subtask which
00:09:34.240
can again create a sub sub task etc
00:09:39.200
another thing to clear out is it's all
00:09:41.360
just ruby
00:09:42.800
async does not contain any special dsl
00:09:46.000
nor does it do gimmicks like monkey
00:09:47.920
batching
00:09:50.000
previous example performs only http
00:09:52.560
request within tasks but you can run any
00:09:55.519
ruby code anywhere
00:09:57.360
main tasks or subtasks
00:09:59.680
it's just regular ruby code with method
00:10:02.240
calls and blocks
00:10:06.640
okay hopefully you had a positive first
00:10:09.200
impression of async ruby
00:10:11.040
once you get a little used to how things
00:10:12.640
work you see it actually it's really
00:10:14.640
neat
00:10:15.839
and the performance benefits are awesome
00:10:18.880
let's now see another code example if
00:10:20.959
you're not impressed yet this may just
00:10:23.040
blow your mind
00:10:26.480
you may have not liked we're using a new
00:10:28.640
http client in the first example
00:10:31.279
the truth is you can use ruby's uri.open
00:10:34.959
to achieve the same result
00:10:44.320
here we see that two requests triggered
00:10:46.399
with uri that open are completed in
00:10:48.720
about two seconds same result as before
00:10:54.079
but uri.open may also not be your
00:10:56.480
favorite tool
00:10:58.000
the brilliant thing about async ruby is
00:11:00.720
that any http client is supported
00:11:04.399
let's let's use http party and see how
00:11:07.279
that works
00:11:18.640
okay the program ran in about two
00:11:21.040
seconds which means all requests ran
00:11:23.440
concurrently
00:11:26.800
so far we've only seen examples making
00:11:29.440
http requests but what about other
00:11:32.000
network requests
00:11:33.839
let's try redis which has its own
00:11:36.160
protocol built on top of tcp
00:11:42.000
this redis command runs for 2 seconds
00:11:44.399
before returning
00:11:52.480
we run the program and it completes in
00:11:55.040
about two seconds so
00:11:56.720
wow we can also make reddish commands
00:11:59.040
asynchronous
00:12:00.959
in fact
00:12:02.000
any io operation can be can be made
00:12:04.720
asynchronous
00:12:06.639
all existing synchronous code is fully
00:12:08.959
compatible with async
00:12:12.240
you don't have to use async only gems
00:12:14.880
like asynch http or asyncretics you can
00:12:18.079
just continue using the libraries you're
00:12:20.240
already familiar with
00:12:24.800
let's add another example to the mix
00:12:26.880
i'll use net ssh to execute an ssh
00:12:30.240
command on the remote server
00:12:32.800
this ssh command runs sleep on the
00:12:35.040
target server and it runs in about two
00:12:37.279
seconds total
00:12:46.240
okay
00:12:47.120
there you have it we added ssh to the
00:12:49.519
mix and it works seamlessly with other
00:12:51.519
network requests
00:12:56.560
you may be wondering what about
00:12:58.240
databases we connect the databases over
00:13:01.040
the network
00:13:03.360
i'll use sql the gem to check if
00:13:05.600
asynchronous database operations are
00:13:07.600
supported
00:13:09.040
the query you're looking at takes
00:13:10.560
exactly two seconds to run
00:13:20.880
and yes
00:13:22.079
that is supported as well cool right
00:13:28.000
let's see another example
00:13:31.920
what do you expect this sleep will do
00:13:34.000
will you increase the total program
00:13:35.600
duration by two seconds
00:13:44.880
the whole program runs in about two
00:13:46.880
seconds which indicates this sleep ran
00:13:49.440
concurrently with other tasks
00:13:52.399
nice
00:13:53.440
so not only can we run network io
00:13:55.839
asynchronously we can also run other
00:13:58.160
blocking operations async
00:14:01.279
what other often used blocking
00:14:03.120
operations do we run in ruby
00:14:05.839
how about we try spawning new child
00:14:07.760
processes
00:14:10.639
i'm using a sleep system command in this
00:14:12.639
example
00:14:13.839
don't get confused this is actually
00:14:16.000
running an external system command it
00:14:18.480
could be any other executable
00:14:20.800
i chose sleep so i can easily control
00:14:22.800
the duration
00:14:30.959
and there you have it system commands
00:14:33.199
can run async as well
00:14:37.519
we covered a lot in this last example
00:14:39.680
and hopefully these features look
00:14:41.600
exciting
00:14:42.880
you saw something new something really
00:14:44.880
innovative in ruby
00:14:46.480
but that's not all
00:14:48.000
let me show you how easily does async
00:14:50.000
ruby scale
00:14:53.279
i will run every task in an async block
00:14:55.839
10 times
00:14:57.519
quick note about net ssh
00:15:00.000
i had to remove that one because i
00:15:02.000
couldn't figure out the correct ssh
00:15:03.760
configuration for this example
00:15:06.480
what do you think how long will this
00:15:08.160
program run
00:15:17.120
two seconds yes
00:15:19.040
we're running 60 tasks each last two
00:15:22.079
seconds and total program runtime is
00:15:24.480
slightly more than 2.5 seconds
00:15:28.480
how about cranking things up
00:15:30.399
how about we repeat this a hundred times
00:15:37.440
let's see what happens
00:15:48.240
we're now running 600 concurrent
00:15:50.480
operations
00:15:52.000
the total program runtime increased by
00:15:54.320
second because of the overhead of
00:15:55.920
establishing so many connections
00:15:58.480
still i find this pretty impressive
00:16:03.120
so there you have it easy scaling with
00:16:05.519
async you can crank the numbers up but
00:16:08.079
in my case redis server and postgres
00:16:10.079
database started complaining so i left
00:16:12.320
it at that
00:16:14.079
and you can argue we could do the same
00:16:15.759
thing with threads creating 600 trades
00:16:19.040
i think that's really pushing the limits
00:16:20.720
with threads my my hunch is that thread
00:16:23.839
scheduling overhead would be just too
00:16:25.759
high
00:16:26.880
when using threads it's more common to
00:16:28.959
limit the number of threads to say 50 or
00:16:31.680
100.
00:16:32.880
on the other hand
00:16:34.320
600 concurrent async tasks is a common
00:16:37.440
thing to do
00:16:39.199
the upper limit on the number of async
00:16:40.800
tasks per process is single digit
00:16:42.959
millions
00:16:44.079
some users have successfully done that
00:16:47.199
this limit of course depends on the
00:16:48.959
system and what you're trying to do
00:16:51.199
for example if you're making or
00:16:52.639
receiving network requests you're
00:16:54.639
probably run out of ports at 40 50 000
00:16:57.759
concurrent tasks unless you play with
00:17:00.000
your networking
00:17:02.560
in any case i hope that you get the idea
00:17:04.720
that easy ruby is a very very powerful
00:17:07.360
tool
00:17:10.160
to me the biggest magic is running three
00:17:12.319
http requests with the uri.open
00:17:15.600
with vanilla ruby that takes 6 seconds
00:17:18.480
and then by using the same method within
00:17:21.199
async block the program runs for 2
00:17:23.679
seconds
00:17:25.039
same with other examples sleep relis etc
00:17:29.200
they all normally run in a blocking way
00:17:31.760
but then inside an async block they work
00:17:34.559
asynchronously
00:17:36.559
it's a great example of keeping ruby
00:17:38.640
code fully backwards compatible
00:17:41.520
but how does that work
00:17:44.640
there's a lot to learn about async ruby
00:17:46.720
but i think there are three main
00:17:48.080
concepts to understand
00:17:50.960
event reactor
00:17:52.559
fibers and fiber scheduler
00:17:55.360
each of these three topics is very broad
00:17:57.679
so i'll just provide a summary
00:18:01.520
let's start with the event reactor
00:18:04.080
event reactor is sometimes called other
00:18:06.080
names event system or event loop
00:18:09.360
every async implementation in every
00:18:11.360
language say javascript always has some
00:18:14.240
kind of invent reactor behind it
00:18:16.880
async ruby is no exception
00:18:19.440
current version of async gem uses neo4r
00:18:22.000
gem as an event reactor backend
00:18:25.280
neo4r then uses libv to wrap systems
00:18:28.000
native apis
00:18:29.679
e-ball on linux kq on mac etc
00:18:34.240
what does the event reactor do
00:18:36.799
it effectively waits for i o events
00:18:39.760
when an event happens it performs an
00:18:42.080
action we programmed it to do
00:18:44.559
on a very high level
00:18:46.559
we make an http request and then we wait
00:18:50.000
event reactor can notify us when the
00:18:52.480
response for that request is ready and
00:18:54.640
can be read from the underlying socket
00:18:57.360
these notifications are very efficient
00:18:59.440
with resource usage and allow high
00:19:01.760
scalability
00:19:03.200
for example if you hear a server can
00:19:04.960
handle 10 000 connections at the same
00:19:07.120
time
00:19:08.000
or a crawler can make a large amount of
00:19:10.080
concurrent requests
00:19:11.600
event reactor is probably the technology
00:19:13.760
behind that
00:19:18.240
you saw async has tasks
00:19:20.640
tasks are just wrappers around fibers
00:19:23.840
event reactor drives the execution of
00:19:26.080
these fibers
00:19:27.440
for example when response in task 1 is
00:19:30.240
ready event reactor resumes task or
00:19:32.960
fiber number 1.
00:19:34.720
later on when response in task 2 is
00:19:37.120
ready it resumes task or fiber number 2.
00:19:40.080
you get the idea
00:19:41.760
due to the decision to register fibers
00:19:43.919
with event reactors we get a really nice
00:19:47.280
property that code within a single task
00:19:50.080
behaves completely synchronously
00:19:52.799
this means you can read it top to bottom
00:19:56.160
this is huge it means our async programs
00:19:59.120
are easy to write and understand
00:20:01.600
the code behaves asynchronously only if
00:20:04.480
you use
00:20:05.480
task.async
00:20:07.440
there's no way you can get callback hell
00:20:09.840
with async ruby
00:20:13.679
the last piece of the puzzle and the
00:20:15.440
last big concept is fiber scheduler
00:20:18.320
fiber scheduler was listed as one of the
00:20:20.480
big ruby 3.0 features
00:20:23.039
it provides hooks for blocking functions
00:20:25.200
inside ruby
00:20:26.880
examples of those blocking features are
00:20:29.760
waiting for an i o read or write
00:20:32.480
or waiting on a sleep function to finish
00:20:35.840
in an essence fiber scheduler turns
00:20:38.159
blocking behavior into non-blocking
00:20:40.320
behavior inside an async context
00:20:44.159
let's take sleep method for example
00:20:46.320
if you're running sleep 2 in an async
00:20:48.480
block instead of blocking the whole
00:20:50.559
program for 2 seconds the fiber
00:20:52.640
scheduler will run that sleep in a
00:20:55.039
non-blocking fashion
00:20:57.440
it will use event reactors timing
00:20:59.360
features to effectively sleep in one
00:21:01.200
task while allowing other tasks to run
00:21:04.159
during that time
00:21:06.799
so that is the big benefit file of fiber
00:21:09.360
scheduler along with fibers and event
00:21:11.919
reactor it makes ac ruby seem like magic
00:21:17.600
okay
00:21:18.559
now that we have a high level idea how
00:21:20.559
things work it's time for the big
00:21:22.320
question
00:21:23.520
does it work with ruby on rails
00:21:26.559
the answer is currently no
00:21:28.880
the reason is active record needs more
00:21:31.039
work to support async gem
00:21:33.600
another big question that you may have
00:21:35.200
is
00:21:35.919
is async ruby production ready
00:21:38.559
the answer to that question is yes
00:21:41.200
async ruby is production ready and a
00:21:43.520
number of people are running it in
00:21:45.039
production
00:21:46.559
everyone using it says nothing but
00:21:48.159
praising praises about async
00:21:51.840
recently i've talked to trevor turk from
00:21:53.840
helloweather.com
00:21:55.679
they replaced puma and typhus hydra with
00:21:58.080
falcon and async http
00:22:00.320
they immediately cut their server cost
00:22:02.159
to one third and their overall system is
00:22:04.720
now more stable
00:22:08.000
if you're excited about what you saw in
00:22:09.679
this speech you're probably asking how
00:22:11.919
do i get started
00:22:14.159
i think the single best starting point
00:22:15.919
to learn async ruby is asyn github repo
00:22:18.960
from there you'll find a link to project
00:22:20.799
documentation
00:22:23.679
i've already mentioned samuel williams
00:22:25.840
but i think it doesn't hurt to say this
00:22:27.440
again
00:22:28.799
this guy is the sole creator of async
00:22:31.200
ruby async ecosystem and a ruby core
00:22:34.159
committer that implemented fiber
00:22:36.159
scheduler feature
00:22:38.240
huge thanks to samuel he's making an
00:22:41.039
awesome contribution to all of us ruby
00:22:43.120
developers
00:22:47.919
i hope you like what you saw in his
00:22:49.600
speech
00:22:50.720
async is an exciting new addition to
00:22:52.720
bluebee it's a whole new type of
00:22:55.039
concurrency added to the language
00:22:58.000
as you saw it's super powerful and very
00:23:00.559
scalable
00:23:02.400
this changes what's possible with ruby
00:23:04.880
it changed the way i think about
00:23:06.480
designing programs and apps
00:23:09.280
one of the best things is it does not
00:23:11.440
obsolete any of the existing code
00:23:14.320
just like ruby itself async gem is
00:23:16.880
beautifully designed and a joy to use
00:23:19.760
happy hacking with async ruby