00:00:12.240
and thank you very much for uh being here there are so many other interesting talks and but you choose to come over
00:00:17.760
here oh um okay so i'm shashank darth
00:00:23.279
i this is my hobby i do it for fun and loss um yeah because i have lost a couple of
00:00:29.119
drones and never got them back and i haven't made a time out of them yet for the doors to fly though we need a little
00:00:35.040
bit of energy in this room so here is what we i want you guys to do see this slide over here
00:00:41.440
on the count of three i want you all to speak as loudly read that thing as loudly as you can so that the energy in
00:00:48.320
this room goes up okay all right one
00:00:53.520
two three
00:01:00.079
thank you all right now if you have blessed me because i'll show you why that blessing is required
00:01:07.040
and here pretty soon these two should take off using my hello
00:01:13.439
edu dot rb file i have to do one little thing that i
00:01:19.040
forgot give me just a sec i have to find the ip address of these two guys when they booted they got a
00:01:25.759
different ip and the way i communicate with them is there is a little uh
00:01:31.439
wi-fi router that i have set up over here and my laptop is connected to them through that so i'm going to scan for
00:01:37.680
these devices and it should take not more than few seconds for it to detect what the ip
00:01:43.920
address is of these two guys and it's telling me that it is 102 103
00:01:49.520
the last three digits that is so i'm going to give this a shot okay if our energy is right they're going to take off and land not
00:01:56.320
exactly the same spot but somewhere if they start drifting i have drone catchers in the audience who will catch
00:02:02.719
them okay want to okay let's go yep
00:02:08.560
let's see if it turns green there you go
00:02:24.239
that's like my teenage daughter who doesn't listen to me uh she has a knife
00:02:31.519
right it will land in a few seconds it has a default 15 second
00:02:44.560
they were much well behaved today than they have been in the past so let's see how that goes i have to be in
00:02:51.440
the present moment now right okay cool so uh these are a space special
00:02:57.840
class of drones called teledrones tello is the type of drone that i have been programming with uh these are probably
00:03:04.879
the cheapest that you can find that are programmable they're sub 150 each uh the first few that i lost i was
00:03:10.959
testing how high they can fly and then the drift of wind came through and they were gone with the wind
00:03:25.760
they introduce beginners to programming uh you must have if those of you attended the previous talk he mentioned
00:03:31.920
the language called logo which was a language to teach kids how to draw turtles or diagrams using turtles on a
00:03:38.159
2d plane this is a 3d version of that you know you can not only go this way but you can go up and down
00:03:43.840
so the purpose of these drones is to teach people how to program so there are many simpler concepts in it but there
00:03:50.319
are also some deeper constructs that you know more more advanced programmers can uh get some benefit out of
00:03:57.439
the type of drones that i'm using over here are called yellow edu type of drones so they have
00:04:02.799
progressed from sdk 1.0 to 2.0 until you edu is what we saw just now um it
00:04:09.680
enables us to program swarm of drones like you saw i i couldn't i did not have the budget
00:04:15.519
to buy the full edu kit which comes with 20 drones and costs about five thousand
00:04:20.560
dollars but i could buy a couple and try these things on my own and these kits they come with uh
00:04:26.400
official support for python scratch and swift i said well uh why not rupee you know so
00:04:32.880
i took it up to myself and and and a friend of mine and we got it all working in ruby and
00:04:38.160
my interest is in my ruby so i got it working in a ruby too they require these concepts of network
00:04:44.080
programming concurrent concur programming you know video streaming and stuff like that so
00:04:49.280
um these are the specs it has an intel 14 core processor in that little thing that
00:04:55.600
you saw flying uh very light weight they have a camera believe it or not
00:05:01.680
they have collision detection sensors in them and the two there are two flight modes available in the edu that you uh saw
00:05:08.639
just now there's a ap mode and a station mode so what's ap mode that's the access point mode that comes
00:05:14.320
by default when you pull it out of box you can't swarm it you have to set it up
00:05:19.680
for forming but the default setup is where you can have one computer control
00:05:24.880
one drone one is to one or multiple computers controlling that one drone
00:05:30.000
but the fact remains is that a single computer can only control one run
00:05:36.560
i have never tried the multiple uh computers controlling one drone um you
00:05:41.680
know in the past but it is possible and then in that mode the video streaming of the camera is on
00:05:47.520
so that's the default mode in which these drones work but then there is the swarm mode or the station mode as they
00:05:52.960
call it and it's only available in the second version of all these drones that led you runs and you have to set that up
00:05:58.960
manually so after you pull it out of the box there's a little bit of configuration you have to do to get it set up in the
00:06:04.639
swarm mode and in that case what happens is you use a router like i'm using over here to
00:06:11.039
communicate with the drones uh from your laptop uh in the previous mode the drone
00:06:16.080
itself was the access point so it publishes its own ssid and its own password and you have to capture it in
00:06:22.160
your program and then control it that way and that's the reason why you cannot flip between those two
00:06:27.840
so these are the two modes of of communication and these are the programming prerequisites for any
00:06:33.280
language to be able to control these these drugs okay so the first
00:06:39.120
requirement is that you need udpip okay communication should be in udpip
00:06:44.800
between these runs the second requirement is that you should have some form of concurrency because you're going to send a command
00:06:51.120
in one thread and receive the response in another thread and you need to know which command does and which response
00:06:57.120
match with each other so you need some notion of concurrencies involved in
00:07:02.479
there i tried sending one after the other like send a command and wait for the response in the same thread never work
00:07:09.759
because udpi is fast enough and we'll see why and then the video streaming is kind of optional i couldn't get it to work for
00:07:16.319
ruby or everybody but if you really want to see the real-time video then you have to have a real-time video processing
00:07:21.759
library in your in your programming language that allows you to do that so how does that work out in
00:07:28.840
ruby um in ruby there's a udp socket class in standard ruby
00:07:34.240
i use that for communication there's fibers and threads in ruby two dot x and then there is a reactors you
00:07:40.720
must refer to fractures in other talks um in c dot x so there are so many ways of doing concurrency ruby
00:07:47.680
for video streaming i was so overjoyed when i saw uh oh i got my
00:07:54.720
i'll i'll come to this gem a little later i got my sequence a little flipped
00:08:00.319
okay so what is udpip it's you must have all heard of tcpip uh udpip is just a
00:08:05.599
brother of pcbf socket based uh the lower the
00:08:10.639
it is based on ip sockets and uses datagrams and port numbers to communicate it is connection less which
00:08:18.240
means no handshaking you know kobe it's safe and all and then no buffering of datagrams so it sends a
00:08:24.639
command and forgets about it okay um no buffering of diagrams at either end
00:08:30.479
but it is fast and it's meant for this real-time communication between objects that want to communicate and but there
00:08:36.479
is no guarantee of message delivery and there is no guarantee of message order so you might send two commands in one
00:08:42.399
order but the responses might come in some different orders so a programmer has to worry about those things when we
00:08:47.600
uh when when that kind of protocol is used you saw an example of that just today
00:08:53.200
when it went one oven took the command to land with the other one just ignored and because the packet got lost in the
00:08:59.600
communication so that's the drawback of udvip but at the same time it is also very fast and
00:09:05.200
that's is the speed of the communication that matters more than uh than these guarantees that you want the
00:09:11.839
guarantees can be built by the programmer not the speed all right so in ruby uh if you had to kind of use a
00:09:19.360
udp example uh this this is the example i'm going to use throughout has nothing to do with drones but just a way for you
00:09:26.080
to know what udp socket looks like you have to require the socket library of ruby you set up the host and the port
00:09:31.920
because that's how it communicates on that host to that port and some message to be sent to that
00:09:37.600
object and then you create a new socket you bind that socket to the host and the port and you send the message across the
00:09:44.320
socket and then you wait for the data now in this particular example it's not threaded as you can see it is waiting
00:09:51.040
for the data to come in because it's sending and uh reply uh a message to itself it's
00:09:56.880
literally sending it to itself so there is no chance of it getting lost because where it's going to go
00:10:01.920
and then if you print the data that's the data that you see come out of it so prerequisite number one was trivially
00:10:07.760
satisfied by ruby in case of concurrency like i said there was fibers there are
00:10:13.600
fibers which is cooperative concurrency which means the two fibers have to to now the programmer has to know that this
00:10:20.959
fiber is going to give up control for this fiber to take it up and so there is cooperative concurrency in case of
00:10:26.800
threads there is preemptive concurrency which means the programmer has no control when when the thread will
00:10:32.560
switch you know when which thread will get control and then there is reactors
00:10:38.079
this is still experimental i believe in 3.0 but pretty soon it might might become core part of the language it's
00:10:44.880
the actor-based parallel execution so if there are multiple cores on your computer then it
00:10:50.560
will actually do things in parallel so these were also trivially met for ruby
00:10:57.200
and then uh coming to my site that i wanted to show so there is a library called ruby open cv
00:11:03.680
which wraps the open cvc c plus library for video streaming my c plus plus is not that great i tried
00:11:10.560
very hard to compile it to bring it up to the current level so it is ruby 2.x
00:11:15.600
and cv 2.4.13 that it that it is stagnated at that point it is on that
00:11:21.600
it's right there for it last eight years and i wanted a more recent version i tried getting it up updated i couldn't
00:11:27.760
so i gave up on the video streaming part of it i couldn't even get open cv to run just the raw open cv to run on my mac
00:11:34.800
which is a light uh a more recent mac so this is not satisfied we won't see a
00:11:39.920
demo of video streaming of the drums uh from their camera
00:11:45.680
what about m ruby so m ruby by default has only fibers there's no threads in my ruby and
00:11:52.079
for for the socket for the communication part you have to compile mrb with the socket
00:11:57.760
gem using it's a core gem so it comes with m ruby uh it goes with the core uh
00:12:03.120
installation of mrb and again if it's not there in ruby it's unlikely for it to be in
00:12:09.200
if not there in ruby it's not unlikely to be in mrb so as of today i don't i have not seen any
00:12:14.480
any attempt to make uh video processing uh streaming possible in mruv
00:12:21.200
uh not all of you are aware of mrb though so a little slide on what mrab is it's a lightweight implementation of
00:12:26.880
ruby and that's my line of interest my subject of interest and the m stands for maybe embedded ruby
00:12:34.240
or modular ruby or minimalistic ruby there are so many ways you can call the or interpret that m
00:12:40.240
uh in terms of features it's a proper subset of ruby everything you can do in mrb can be done in ruby but not vice versa but in terms
00:12:47.200
of deployment architectures it's a it's the other way around you can deploy
00:12:53.360
mrv at more spots and in more environments than where you can deploy ruby
00:12:58.639
uh the the key the nice thing about m ruby is that you can compile your code into a single executable okay with very
00:13:06.800
little dependencies on the library of the operating system and the use cases typically are for bare metal programming there is no operating
00:13:13.440
system available not even a file system available that's where you can do some bare metal programming using a ruby
00:13:20.079
for memory constrained devices which is my line of interest i used to program these uh hardware uh point of sale
00:13:26.639
systems where you swipe your card on you know and then i i was wanting to
00:13:31.760
program those using mrb i failed because it would be still bulkier than what those devices uh need
00:13:39.279
but we have made a lot of progress since i tried it last so it's probably possible now and those devices also have
00:13:45.440
become bigger in size so the latest version of stable mrb 3.0
00:13:51.360
apparently you can launch an app within 100 kb of ram and the devices that i had were requiring less than 64 kb of ram so i
00:13:58.800
doubt that that would have worked even now or but they are making progress this is one of uh mats's
00:14:05.600
favorite uh uh projects and you will see his commits almost daily coming into m
00:14:10.639
ruby it's very active and then of course game engines and and things like this so that's what mr ruby is all about
00:14:16.880
how do you do this in ammunition well the only thing only one line change is that there is no require why no require
00:14:23.360
because it does not assume the existence of a file system and requires require requires files to exist required from
00:14:30.000
where so no require and nothing else changes same code would work
00:14:35.760
if there is no require possible how are you going to do it okay so what you have to do here
00:14:41.360
is you have to recompile the interpreter with that core gem okay there is a build configuration
00:14:47.279
script in mrb when you clone clone the project and in that there is this file called build config slash default rv if
00:14:54.320
you just add this extra line saying hey i want you to be compiled with m ruby and the socket library of mrb that's how
00:15:01.360
you would do it when you get that into the build and you clean up the previously compiled version if you had any by doing rake
00:15:08.240
clean and then rake it it will compile the interpreter with the socket layer built into it
00:15:14.399
okay make sense and then what you have to do is use that compiled version of mrb to run that example so i haven't
00:15:21.199
touched the example i've removed the require and recompiled everything but what if i want to convert this
00:15:26.560
program into a single executable that i can distribute like a exe or you know a
00:15:32.560
binary uh that process is slightly convoluted uh you there are two ways to do it you
00:15:38.800
can either compile your program that example or rb into a byte code
00:15:43.920
by using a mrbc command that gets compiled in the bin directory of mrb when you
00:15:50.959
recompile it and that byte code can then be distributed to whoever has m ruby
00:15:56.160
installed on their system and they won't see the original code source code because now they only see the byte code
00:16:02.079
and then they have to use the minus b option to run that example and gets this it does the same thing
00:16:08.399
but what if you want to go further and say i don't want the person to have m ruby installed on that machine or that
00:16:13.440
device right so that's where a little bit of c is required it's a two-step process so what you have to do is you
00:16:19.839
have to use mrbc the same compiler to generate what is known as a symbol table okay
00:16:27.040
i won't go into the details of that but it's a table that is a data structure that stores the bytecode okay
00:16:32.800
and then you require so you give a name to that symbol table that you want to use there are some conventions
00:16:39.519
not rules convention that you have to follow and use the same example dot rb
00:16:45.279
to emit a file called udp example.c it just takes the the file name uh removes
00:16:51.040
the extension and attaches dot c to it and that's the data structure that you will have to include and then step
00:16:57.839
number two is you have to write boiler plate code it's available in many examples are there you just pick one i
00:17:04.480
picked the simplest one that would fit on my slide here as you can see i did not care about handling errors and stuff but that's
00:17:10.640
obviously not production quality code but what you have to do is you have to include that emitted file
00:17:16.799
and then you have to uh open as if you know opening the vm these are standard steps of compiling any any uh
00:17:24.799
mrbc code and then uh see that line that the dotted line that goes down
00:17:30.240
there that uses the symbol table that you generated in that step one and then at that point that c code that
00:17:36.880
you see on the on the right hand side is a regular c uh file i just use my gcc
00:17:43.760
compiler that i have on my mac use the includes um directly to include
00:17:49.440
the files that are needed libraries that are needed and compile it and that emits the udp example binary
00:17:56.400
that creates the binary file and that's the binary file that you can then distribute to whosoever wants it they don't have to have a ruby on their
00:18:02.400
system you get a nice compact binary file okay so these are the steps to create a native uh what we call bare
00:18:10.160
metal binary all right so that's uh mrb i had three
00:18:15.600
uh focus areas for the purpose of this talk one was to focus on teller one was to focus on actors and fibers and
00:18:22.080
threads and one was m ruby but i chose him ruby because that's my favorite um so okay well show me the code uh how
00:18:31.039
did we get these things to fly uh are they still on oh i should have turned them off
00:18:36.240
so one thing that i have noticed about these is the battery life even though the
00:18:41.280
reported battery life is uh what should i say uh 15 minutes of flight time i've never gotten 15 minutes of flight time
00:18:47.360
on one charge i've only got about five minutes of lime flight time so you have to be careful about these i have a extra
00:18:53.440
pair of batteries so if you run out of batteries that's not a problem uh how much so that's the time that i have left
00:18:59.919
right but here monitor okay so i have um
00:19:04.960
on my github um i've created a project for telu um where
00:19:11.200
i have written the same code that i will show you just now in ruby and mrb and crystal which is a
00:19:17.679
language that i'm playing with and a friend of mine contributed in elixir because remember those three prerequisites video is just optional but
00:19:24.400
those two any language that has those two should be able to draw drive these drones so if you have
00:19:29.919
any favorite language that is not on that list in the project please submit a pr and i would be happy to merge it in
00:19:37.840
we are going to only focus on the second github uh project which is i curated my
00:19:43.679
list of ruby files for this proj for this conference and the one that we saw running right now today just now was
00:19:49.840
hello edu.rb we will go through that and all of this started because of the gentleman called tom black
00:19:56.160
in one of the previous ruby cons he had a five-minute lighting talk when he demoed these and i
00:20:02.400
got excited by what he showed us he used the previous version of it which could not be swarmed so i worked with
00:20:08.880
him and we collaborated on coming up with the second version of the gem my pr request is still pending but i'm pretty
00:20:15.039
sure he will after he likes the code the way i've written the code he will merge it in so if you go to that particular
00:20:22.640
github repo you will get a gem that allows you to stand up a telo server a local taylor server so you
00:20:29.280
don't need the actual tellers to be there and you can test your client code against it and all
00:20:35.360
that kind of fun he has put a lot of effort in it so i'm very thankful towards uh tom black all right so that's
00:20:41.600
the code that we're going to see today and also modify so i'm going to pop out of this and
00:20:47.280
point you to tom black's uh repo that's the telegem that i'm talking about
00:20:53.520
um this is my repo and this is the code that i have written for the conference so please uh and this is the
00:20:59.919
project where i said your pr spears will be appreciated what i have to do
00:21:05.440
now is i am on okay since i am not on the public wi-fi i'm on this wi-fi which is
00:21:11.600
not connected internet i won't be able to drive through so what i'm going to do is instead i'm going to show you the
00:21:16.640
code uh the code that i want to you all to see this is not production quality code this is just so that i can get it
00:21:22.559
in as little this is the hello world equivalent so it's not at all how you should write code okay uh for
00:21:29.280
example uh let me see if i can pump this up can you all see this
00:21:34.480
um so for the two
00:21:41.280
for those two drones that i have over here this is hardwired for two drones and two rooms only i've used
00:21:47.440
global variables you shouldn't do that there is no class i have just used methods that send a command to the two
00:21:53.760
drones and receive the responses coming back from those two drones and i i did
00:21:59.600
all my work on raspberry pi which has a camera attached to it and so as soon as these doors take off it starts the
00:22:05.360
camera on and captures the video that way if things did not work i would be able to show you the video and say hey
00:22:10.799
look this is how it worked but thankfully it worked so this is option completely optional the if
00:22:16.400
statements will take care of it not coming in the way if it's not a raspberry pi and then all you have to do
00:22:21.679
is start a thread you know which will receive the response that you're going to get from the drones
00:22:27.840
send the commands one by one this is to set the telo in the command mode or in listen mode you know when you talk to
00:22:34.000
your kids listen to me listen that's that's that mode you you go in the command mode
00:22:39.120
and then you check the battery level because they are very there are certain gymnastics it can do only when the battery levels are above
00:22:45.520
some values uh for taking off uh for takeoff and landing you don't need much
00:22:51.520
so here you can see i checked the battery i asked it to take off uh i'm doing i'm we are going to do some
00:22:57.440
of these things right now on the fly and then land and then you know mission accomplished and close the close the
00:23:03.520
sockets okay so this is the code that i ran uh i'm going to try it again
00:23:08.720
but this time i'm going to try to see if we do some more with the with them than just
00:23:14.720
take up and land i'm only
00:23:20.840
hoping that there's enough battery for them to do this okay so while they are booting
00:23:28.559
uh the boot sequence what i'm going to try to do is
00:23:33.679
flip them let's see if that works okay all right so
00:23:38.720
one of these drones will flip right the other room will shift left they will go further apart
00:23:43.840
and then land okay that's all i'm going to do here let's give that a shot and see what happens
00:23:50.240
okay so do you see the you know what i should also explain the
00:23:56.000
font size so you can see the running of the of the okay
00:24:01.120
typically when you reboot them if no extra device has been attached the ip addresses retain so i'm hoping that the
00:24:07.679
same two ipads are still valid um and let's see what happens
00:24:13.360
are they all on both saying yellow yeah okay let's see
00:24:37.840
all right i have five more minutes should we keep on going all right all right then let's do one
00:24:43.919
more thing and try to do the flip and flip two flips
00:24:49.440
okay two flips same thing right watch this
00:25:06.320
and now they should come back together if there are no errors on the console
00:25:11.600
but this one did
00:25:26.880
i will have ques i will have i want to do only one more thing and then we can switch over to uh
00:25:34.400
uh question answers okay all right so this what it is doing is
00:25:40.000
going to rotate clockwise one is going to rotate counterclockwise the other one is going to read clockwise and let's see
00:25:45.039
if that works okay so let me clear this can you all read the
00:25:51.600
font i mean is this large okay return all right here you go okay okay i
00:25:58.159
still have views they should kick off they should flip
00:26:05.360
there you go now they should come ready
00:26:12.400
see
00:26:17.600
there you go thank you very much thank you thank you
00:26:29.440
that was it guys this is this is they have been in their best behavior today
00:26:35.679
questions i have three minutes for questions
00:26:40.960
as you can see it's very simple um please please please submit pr prs of of
00:26:46.000
the of the code in uh oh i do want to do one more thing if you do submit a pr i would
00:26:52.240
like for you to follow the model that my friend followed so i'm going to switch over from
00:26:59.200
this mode um
00:27:08.240
you have to drop off and i'm going to now connect to the ruby uh the public internet uh here and if
00:27:16.320
you notice this is mine right yeah okay so in my uh
00:27:22.960
uh codebase i as i said you have crystal which is a language like ruby elixir mlb python ruby what not but my
00:27:30.240
friend craig uh submitted his pr just a few days ago and that's the model i
00:27:35.360
would like to follow none of my code has that much of detail he has given all the details in the readme
00:27:40.480
uh how to install it how to use it that's awesome i would love vrs to be like this tara
00:27:47.039
thank you very much
00:27:54.159
yes can we try that let's try that
00:28:00.320
so i'm going to i have only one minute left so i don't know how much so i'm going to try it
00:28:05.679
with the newer one okay it is possible good question thank you very much
00:28:11.279
because that does not bring out the power of the telegem so here i'm going to go into
00:28:17.520
my telo gem folder if you all can see it just to be on the safe side i'm going to
00:28:23.840
kind of i think rake builds it yeah okay so it
00:28:32.480
ah which was oh
00:28:41.760
ah that's why okay i'm not going to fight this i i have the telogen i believe the edu gem also installed on
00:28:47.760
mine so let's see if that works so this is like a uh interactive mode like irb off of
00:28:54.399
teller and now no
00:28:59.679
sorry guys um so fine
00:29:10.000
does it change dang it i
00:29:16.000
i should have thought of this question beforehand and be prepared with it uh
00:29:26.000
is it saying the permission error anyway so the the telegem allows you to
00:29:31.360
run it in an interactive mode like uh like your question you were asking greg
00:29:36.640
uh it is possible to send it command by command but then you have to remember that the first command you have to issue
00:29:41.840
is the command command and then start with the takeoff flip land and stuff like that um
00:29:48.559
i'm almost out of time here so if you want we can we can try it outside somewhere you know let's capture a
00:29:54.799
little niche and play around with it and you all can come and join and if you install the gem you we will be able to
00:30:00.880
control two computers to control one gem and multiple controllers controlling multiple uh of these tellers and stuff
00:30:07.600
like that but yeah it is possible to to ants to answer your question greg
00:30:14.399
thank you thank you again thank you