00:00:24.680
so I'm going to talk today about row row rumba and that's room bound Ruby on
00:00:30.449
Rails and this is a rather different presentation what you might be used to so I'm going to start off by saying a
00:00:36.989
big thank you to I robots especially nancy and Joshua and I robot for supplying us shipping to brand new
00:00:44.700
rumbas two rails comp for us to use and some demos that we're going to do that was really awesome with them and we just
00:00:51.030
kind of contacted them like three or four weeks ago and say hey we want to do presentation about room and they were more than gracious about sending us the
00:00:56.850
Roomba so thank you very much the next thing I'd like to do is say a big thank you to cook pad because after my
00:01:02.820
presentation was selected my proposal was selected I was trying to figure out how in the world am I going to get to
00:01:08.249
the US and book all the hotels and take care of everything and cook pad did an awesome job just encouraging me and
00:01:14.219
setting up everything for me so that was really awesome that a company that the company that I work for is giving back
00:01:20.549
to the community and in such a big way so big thanks to them so without further ado I'd like to start with some
00:01:27.240
introductions and these rumbas are not the rumors that I've been working with in fact the rumor that I've been working
00:01:33.060
with the most time is still in Japan and this is him this is Roo and you see
00:01:38.520
there he's got a kitchen knife a butter knife attached to him and I did that so that I wanted to arm him so that way he
00:01:44.639
would remember fondly the person that first gave him a weapon when the robot uprising happens I thought one
00:01:53.039
particular loan wasn't actually enough considering how great an awesome rumba was so I opted up the notch gave him
00:02:01.319
some more utensils he looks a little more fearsome this way so that's my Roomba konnichiwa my name is Charles
00:02:08.189
Abbott I work for cook pad I've been there for a little a year in Japan based out of Tokyo I run
00:02:14.940
a little blog called before the code org and this blog has really almost nothing to do with code so it's a cunning
00:02:21.480
oxymoronic but the the blog itself is just my journey moving to Japan three years ago I didn't know anything about
00:02:27.630
Japanese culture a new almost zero language at that time and this is my daily life about all the trials that I
00:02:34.020
faced while living in Japan so if any of you were even remotely interested in Japan please check out the blog I still
00:02:39.180
write at least two or three times a week about new and interesting things that I find so since the 5th or 6th grade one
00:02:47.100
of my favorite quotes was by an author named Daniel J boorstin he wrote this amazing series of books historical books
00:02:54.180
he was actually the historian at the Library of Congress it was they discovers the creators and the secrets I
00:02:59.430
don't know if any of you have read these books but they're really really great and my favorite quote comes from him and
00:03:04.950
he says that the greatest obstacle to discovery is not ignorance but the illusion of knowledge and it essentially
00:03:11.760
means that throughout human history the greatest things that blocked any new paradigm shift where the fact that
00:03:17.700
people were holding on to old ideas or holding on to what they presumed was only the right way to do something and I
00:03:24.990
think this this quote still has a lot to do with me on a daily basis and especially with this presentation as
00:03:30.810
well because five six months ago when I started out on this hacking Roomba journey I knew absolutely nothing about
00:03:36.540
physical computing I knew nothing about wiring I had never done soldering I've never done anything I'm going to talk to
00:03:42.060
you about today and so ignorance was my best friend and I hope that throughout
00:03:47.730
through this journey in this presentation I can at least encourage any of you they have never done anything like this to step out and just try it so
00:03:56.750
when I first got my Roomba about six seven months ago I purchased a Roomba
00:04:02.700
and I opened it up and the packaging around the Roomba had this sticker which
00:04:07.739
was really awesome it says this robot has an API it can be hacked and if you open this bag you are accepting the
00:04:14.459
end-user license agreement that says your room but can be hacked and we encourage you to do so i rememba is really awesome about the open interface
00:04:21.720
that they have and so this particular rumba here I took the
00:04:27.950
sticker off and paste it down here so I remember the end user license agreement
00:04:33.200
while I'm hacking and running my Roomba round about four months ago at the Tokyo rails meetup that we host there was a
00:04:41.150
call for any ideas for projects that anyone might have and I decided to come
00:04:46.910
up come before the Tokyo girls made up and see if anyone was interested in
00:04:51.980
hacking rumbas with Ruby and my presentation at that time was called three r's and essentially it was I have
00:04:58.040
this idea and I want to use Ruby I know that there's a serial port gem that we can use and I want to use Ruby and I
00:05:05.450
want to use rails and I want to do something with both of those and control my Roomba just to prove that it can be
00:05:12.020
done and the idea was hey you got serial port Ruby Ruby and will control rumba
00:05:18.290
you can have a real site that's going to direct all your requests to your room but remotely something's going to happen
00:05:23.360
in a profit right that's the quintessential equation at the time I really didn't have many resources in
00:05:30.050
fact I had only come across a couple blogs that said that hakki rumbas was like doable it's something that's not
00:05:36.740
incredibly difficult other than just trying to figure out the API etc but
00:05:42.500
every package and every project that had been created for hacking rumbo hacking
00:05:47.840
roomba switch there weren't that many there were only a couple we're all written in Java or some other language there was nothing available out there in
00:05:53.900
Ruby so I decided to get started and the
00:05:59.540
first place to start was to try to figure out how do I actually connect to the roomba and I looked at the documentation and come to find out
00:06:06.200
Roomba has a serial port it's a 7 pin serial port and I had to go out and find
00:06:12.260
the cable that would match a 7 pin snowboard underneath the roombas handle
00:06:18.020
there's the serial port this is probably really impossible for anyone to see but I'm just going to point it out on the newer models there's a handle on that
00:06:24.080
means there's the actual serial port some of the older models these serial ports are hidden underneath covers that you have to pop up pop off the top of
00:06:30.920
the Rimbaud so where do you get a 7 pin serial adapter well one of the best
00:06:37.330
replacements for a 7 pin serial adapter is actually an 8-pin serial mac printer
00:06:43.990
cable and i spent a couple hours in Akihabara if any of you have been to
00:06:50.699
Tokyo Akihabara is a section of Tokyo that's referred to as electric city
00:06:55.960
because there's just hundreds and hundreds of stores of used and new computer equipment and for instance this
00:07:01.990
particular Apple used mac store that I went to had just giant bins of old stuff
00:07:07.810
cables and everything that you could dig through and after a couple hours of digging I found one of one good cable
00:07:13.150
that I could use and then something crazy happened like well three weeks
00:07:19.030
after i got this cable and started hacking on marimba my firstborn child came into the world and suddenly all
00:07:26.289
that free time that i thought i was going to be hacking on rubra somehow disappeared and i never expected that I
00:07:31.750
would really be sitting up like every two hours every day of the night that's incredible but anyways back on track not
00:07:39.219
long after well about a month or two later I finally picked up that Arduino that I had bought and picked up that
00:07:45.789
cable again and I said I've really got to do something with this rumba and I started looking at the Arduino layout
00:07:51.279
and trying to figure out how do i use the Arduino and connect it to the Roomba
00:07:56.490
and Arduino and the top right there are two pins is the receive and the transmit
00:08:02.379
pin that was long and then in the bottom there is some power connectors for the
00:08:09.610
9-volt in the ground and that's what that's all that you need to connect with rumbas to have a transplant receive line and the 9-volt in the ground now I'm
00:08:18.069
going to save you guys some time if you decide to go out in the hat room but there's two sets of documentation out
00:08:23.500
there there's they're both published by iRobot there's the one on the Left which is the Roomba open interface or oh I or
00:08:31.300
ROI and then one on the right is the iRobot Roomba cereal command interface SC I specification the one on the right
00:08:38.769
looks awesome the formatting is grey they've got pitchers in there and it just and these are both the front page
00:08:44.740
covers but and actually the one on the left is the newest documentation
00:08:50.030
and don't even look at the one on the right unless you just want to see some pictures so what is the ROI Roomba open
00:08:58.430
interface or API I'm going to probably refer to the ROI Roomba open interface faces just an API from now on so a we
00:09:05.060
don't get this confused with return on investment or anything but the API is specified in their documents that for
00:09:12.110
instance this is a page from the API that specifies some sensors that you can get from Roomba there's a bumps and
00:09:17.300
wheels drop sensor there's a wall sensor to tell you if there's a wall coming up there's one that tell you if there's a cliff ahead so that we room but doesn't
00:09:24.350
fall off so your stairs or anything like that and it just tells you what is the ID of the packet for instance bumps and
00:09:31.430
wheels drop is packet id7 and it tells you that it's a single data byte and it's an unsigned if for instance with it
00:09:38.060
bumps and wheels drops which is the top there each bit in that bite represents some different sensor reading wall on
00:09:45.440
the other hand is just a single data byte and it's unsigned and it just has a value arranged value of 0 to 1 some of
00:09:51.920
the other ones may have range values to 255 or they might be to bite two bytes
00:09:57.230
that represent a 16-bit signed integer so this is the documentation and after
00:10:03.860
referring to the documentation I was able to wire up roomba wire up the Arduino excuse me this is the cable
00:10:09.560
after I'd cut it and pull out the wires and connected it to the Arduino connected to the power and to receive in
00:10:15.590
the transmit lines and this is when I connected the Arduino to the room but as
00:10:20.690
soon as you connect any Arduino or external device into the room but serial port if it's connected if it's properly
00:10:27.500
set up the room will light up and recognize that there's some device attached to it but he won't do anything other than just light up so what do I
00:10:36.200
have I actually at this point I had the idea that I was going to use Ruby and I
00:10:42.650
was going to send commands to Arduino and then Arduino would somehow forward it to rumba and rumba only communicates
00:10:50.570
on the bite level and then Arduino I really wasn't sure what was happening with our dueno at that time but come to
00:10:56.780
find out it's not so easy to just pass commands from Ruby straight to rumba using Arduino
00:11:03.150
in actuality you end up writing what are called Arduino sketches and there might be quite a few of you out there that are
00:11:08.400
that are very familiar with Arduino and how this works but on an Arduino sketch
00:11:13.500
is essentially just C code and it has two method two functions that run first is the set up where you set up all your
00:11:20.250
pins that are on the Arduino that you can be using to receive and transmit lines tell it what other additional pins you want to be able to use for see your
00:11:27.390
serial connections etc and then you had then it passes off to the loop command which just runs your actual code that's
00:11:33.630
going to tell your device to do something and it runs that code as fast as it possibly can so if you wanted to
00:11:39.090
run it up periodically every one second or two second you have to sleep the code or do something to make sure that it
00:11:44.520
doesn't loop too fast once I started looking at this I said there's something
00:11:50.280
really wrong here I set out to do Ruby and why am I looking at seek code this is not really that cool so I started
00:11:56.010
googling for other options in their turn as it turns out there is a project called rad and it was demoed at ruby
00:12:03.870
conference i believe about five years ago some of you here might be familiar with it but rad is revered we know
00:12:10.350
development and I was pretty excited to see this until I checked the Google mailing group and there's this guy with
00:12:17.070
this awesome name called Lenin Jesus and he asked yes is the project still is the
00:12:23.460
project development stopped and the guy replied and said yeah for the most part it is but you can check some branches and see if you can use it now the Ruby
00:12:30.030
Arduino development the rad project was pretty cool because it would you would write all your code in Ruby as opposed
00:12:36.240
to see on the Arduino and you would write all your Ruby code and then you'd run like a rake task something that was
00:12:42.090
similar to a rake task and it would take your seat and convert it to take your ruby code converted C code Kampala and
00:12:48.270
put it on your Arduino for you now the reason that it's it was not going to be
00:12:54.450
a good option for me is that the Arduino Uno which is the Arduino one point oh it just came out when i bought my board so
00:13:00.600
it was brand new a lot of the libraries that the uno was using had changed and I
00:13:06.120
feared that I would be debugging a lot of compiled seed code that I had not actually written and that was going to
00:13:12.000
be a pain so I sucked it up and decide to write some C code and I'm not going
00:13:19.300
to talk about this very much but this is just an example rumba sketch where I do the setup method where I tell tell the
00:13:26.590
Arduino I'm going to be talking to rumba on a particular pin and I'm going to be talking to him at the baud rate of 115
00:13:33.330
115 thousand two hundred and then I tell him to write some particular commands room but to set it up and in this
00:13:39.850
particular sketch I also designed it so that maybe I could pass commands to the
00:13:45.400
Arduino on a different set of pins from Ruby and then the argument would pass
00:13:50.740
those commands to rumba this was actually very very painful for me I did not enjoy a whole lot because debugging
00:13:58.510
are debugging the Arduino when using it with the Rumba was not very easy specifically when you connect the
00:14:04.740
Arduino to the Rumba it gets his power source from the Rumba and it runs its
00:14:11.020
code constantly and sometimes Ruby the room boat would light up and do something sometimes it wouldn't do what
00:14:16.810
I thought it was supposed to do maybe there was I'd written some bad seed code and so I have to unplug it for the Rumba
00:14:22.120
and then plug it into the USB adapter on my macbook and try to find out what was
00:14:27.280
wrong with it and debugging was just a little painful for me mainly because
00:14:32.770
when you're working with the Arduino you would have to disconnect it from your macbook after uploading the sketch and
00:14:37.960
compiling it and then connected to the room and see if room but does it everything it's rather headless and another issue that I had were some
00:14:44.380
documentation was because it would had just gotten to Arduino Uno some of the libraries to change and I didn't realize
00:14:50.770
what was actually happening so a lot of the code examples out there were not up to date so I was able to turn rumba on
00:14:59.230
but how the hell do I get past first base and that's a big question that I had I really wanted to do something a
00:15:04.510
lot simpler than writing C code and trying to debug it headless Lee whether it as it turns out there's actually a
00:15:10.210
lot of really easy ways to connect to rumba as I've already said we connect
00:15:15.550
with an Arduino microcontroller please excuse my horrible art Arduino microcontroller there's another option
00:15:21.670
that you could actually add an XB wireless adapter to the Arduino and this actually solves one of my problems of it
00:15:26.950
being headless the wireless adapter on the we know on the Arduino would allow me to connect to the Arduino from my macbook
00:15:34.780
while it's while the Arduino is connected to rumba and sending rumba command subway i can get at least output to see what is being sent to the Roomba
00:15:41.410
so I can know what what Roomba is actually getting another option is Bluetooth you can get a serial port to
00:15:49.240
bluetooth wireless adapter connected to rumba and then just sync with your macbook and control rumba another option
00:15:56.020
is straight serial USB to serial cable you can there's an adapter for that you just run cable straight to the Roomba
00:16:02.050
and perhaps the most awesome option out there which I've which I discovered is a
00:16:07.600
Wi-Fi adapter you can connect a serial to Wi-Fi adapter which will allow rumbas
00:16:13.740
remember the wife this is called a wife fly the particular device I'm referring
00:16:18.850
to would get its own IP address from the router automatically and then that way you can communicate it communicate with
00:16:24.940
it from any particular device it doesn't have to be just your one mac book that you sink to synced it to or connected it
00:16:30.430
to so where do you start when you've got something like this well the best place to start is always with the simplest get
00:16:35.950
rid of all the possibilities for something going wrong on the network and just go straight with the cable and that's what I did and that's when i
00:16:42.400
finally got to writing some ruby code thank you this is a my initializer
00:16:50.290
it's not all that pretty but it's for a room booked class that i was writing a room but in ruby and the initializer is
00:16:56.830
pretty simple it just takes a poor or the address on your local computer where
00:17:01.930
the serial serial port shows up you sleep it for a couple of two tenths of a
00:17:07.360
second to allow room but the boot up in cases booting up then you send them a particular start and control code as it
00:17:16.540
turns out writing anything to rumbas is most of the commands that you send a room bar pretty basic all you really
00:17:22.060
sending to it are just up codes for instance if you're wanting to tell him to start up or to do a particular clean
00:17:27.430
session if you want him to clean the floors you would send only just a single integer it converted to a single bite
00:17:33.550
for instance this aph api setup star was just me modeling the Roomba api
00:17:39.400
documentation in room in Ruby code it's not all that amazing but you're just writing a particular
00:17:44.580
alpha code 128 which tells room but to get ready to be put under control run 130 tells them okay i'm going to take
00:17:50.490
control of you now + 135 when you write that there at the bottom it tells them hey i want you to clean the floors and
00:17:56.100
he would end up cleaning now some of the other methods or Roomba open interface
00:18:01.920
commands are a little more complex for instance the drive command if you want to take control of rumba and actually
00:18:07.050
drive him yourself you would have to send a 137 opcode and then after that you would have to send for additional
00:18:14.460
bytes and the first two bytes are actually combined on the room but to make a 16-bit signed integer that room
00:18:21.870
but uses to calculate the velocity for the wheels and this last two bytes that you send our combined on room but again
00:18:28.050
to make a 16-bit integer so some of these would require different numbers of
00:18:34.160
parameters and another API command is
00:18:40.290
called query list that's listed in the documentation of it can take an unknown number of bytes and essentially just
00:18:46.710
right opcode 149 to room but you're just sending a command output 149 hey I'm going to tell you I want you to send me
00:18:53.190
a query list of some particular sensors and the number of sensors that I want
00:18:58.290
you to give me are maybe seven sensors so you would send seven bytes next and
00:19:03.720
then those seven bytes would be the the particular sensor ID for each of the
00:19:10.620
sensors that you want like bump and wheel drops or is there a cliff ahead or do you see a wall or anything like that
00:19:15.710
and then as soon as you write it you would have to sit there and you enter first you have to wait for at least 15
00:19:22.530
milliseconds so that way Roomba has enough time to gather all those sensors and put those on this cereal cereal out
00:19:27.900
port and then you would read from rumba and the bottom of the barrel which is
00:19:32.970
essentially the right command is just writing taking all your all your bites and converting them to single bit
00:19:38.970
character single byte characters and printing them out to the Roomba and the read command is just pretty basic it
00:19:46.290
takes your serial connection sets a time out to make sure that it doesn't hang
00:19:52.050
forever and then it tries to read as many bites from rumbas it can offer all the sensors that maybe
00:19:57.480
roomba sending data back to you with four and so finally pulling it together you would look in your dev and find your
00:20:04.379
your serial device you jump in rails console and you would connect to it i'm going to do a quick hello world demo
00:20:10.980
with rumba now usually in physical
00:20:18.269
computing hello world in physical computing is just turning on a single
00:20:24.570
LED light but i'm going to do a hello rumbo which is oh I need to get in rails
00:20:36.809
console
00:20:43.279
so I'm going to say rue equals rumba new up actually I don't want to do that
00:20:49.769
right yeah rumba rumba done news sorry
00:20:55.399
I've already looked in my dev and found my address to my Roomba and this should
00:21:04.229
return a connection to my Roomba or maybe it won't this is what you get for
00:21:10.950
doing live demos right I think Roomba
00:21:16.679
was sleeping on me okay
00:21:36.130
ok so now Roomba has returned an object and now i can tell roo hey rube I want
00:21:42.040
you to I'm just going to use the API commands which are not abstracted yet I'm going to tell him to drive and I
00:21:48.520
want this is how you would tell a room but to drive you'd send them two bites and for instance I'm going to tell him I want him to drive it in a positive
00:21:54.700
velocity and I don't want him to turn it off and now roomba starts moving now if
00:22:03.910
you notice rumbas not stopping even if I'm pushing him or doing anything he's just going to keep going and that's the
00:22:10.210
interesting thing about this drive the interesting thing about this drive
00:22:16.180
command is that it only tells him to move right and this brings me to the
00:22:27.400
first pitfall forgetting to say win if you doing anything with Reuben you're telling him to start moving you have to
00:22:33.850
know when to tell him to stop and you have to just constantly ask rumba what
00:22:38.890
do you see have you seen anything if you hit anything and if robe of replies with that that's when you have to tell him to
00:22:44.320
stop he's not going to do it by himself when you're in full control mode so how do you get information from him well you
00:22:50.200
asked him for status reports and here on the left is just as gigantic hash that I mapped out trying to map out all the
00:22:57.490
different sensors that Roomba has that you can access and this table here is from the documentation and there's just
00:23:03.520
large groups of sensors I think there's a close to a 30 or 40 different sensors
00:23:08.890
that you can get from rumba a lot of these have different values and like the
00:23:14.020
bumps and wheel drop brings back a bunch of information on has any of the wheels any particular wheel dropped etc I'm
00:23:22.900
going to quickly digress into something that that was a bit difficult for me and
00:23:29.230
that was dealing with the signed and unsigned values coming from rumba a lot
00:23:34.690
of the packets or sensor packets come with like data bytes that are two two
00:23:39.760
bites and they're unsigned why they may be signed and for me an interesting problem or something I've never really
00:23:46.270
experienced with working in Ruby at that time was dealing with binary and signed integers and some of
00:23:52.030
you may be familiar with this much more so than me but the way that that Ruby shows or handles in signed integers is
00:24:02.050
that if you were to convert like negative 500 to the binary
00:24:07.240
representation it seems to just take the bits and then add a negative sign on the
00:24:12.730
front which was kind of interesting to me so the reason why I had to deal with
00:24:17.800
binaries because I have to pull back say for instance two bites from roomba read
00:24:23.770
those two bites from rumba and then combine those two bites to get a single integer value from rumbas that way I
00:24:29.980
know what is the sensor actually telling me I ended up having to dig into some
00:24:35.520
working with two's complement and bit shifting and I've never done this before it was rather interesting and actually
00:24:41.740
kind of fun but I don't know if I want to do it much more than once i also want
00:24:47.410
to digress and say something cool about working at a cook pad specifically was i had a problem with hex and I with one
00:24:55.180
ruby 1.8 point seven and I thought it would it was just a really strange that ruby 1.9 point 3 did was completely
00:25:01.960
different and i sent out an email to all my co-workers as I hey guys what look at this output in Council what the heck is
00:25:07.900
going on here and I swear it was like five minutes later went out the song replied it's like oh that's a bug here's
00:25:14.260
the Japanese mailing list blah blah blah and he was like tell me everything of what was going on us and wow that's so
00:25:19.660
cool that I can get a reply like five minutes from a co-worker one of the guys that's a core committer so so moving on
00:25:27.760
I've already showed that we can do API drive and that's not really all that sexy what I wanted to do was abstract
00:25:34.270
that to a move command or something like that because I don't want to tell roomba like all these really crazy velocity
00:25:40.030
high velocity low I just want to tell room but I want you to move what three feet move three feet in stop that's as
00:25:45.760
simple as it gets right and to do this you just make a method you for instance
00:25:51.820
this define I'm defining move and I tell it's a distance since we already know that we have to send a velocity velocity
00:25:57.610
to rumba the simplest thing is the if you want to tell them you want to
00:26:02.890
move three feet on this in this instance everything is in millimeters with rumba so you tell them you want to move 100
00:26:08.740
millimeters you can calculate with the velocity how much exactly how much time that's going to take and then your code
00:26:14.260
can sit there and loop and check sensors during that time and as soon as it's it's taken X number of seconds to reach
00:26:20.740
that distance you can tell you can tell rumba to stop moving so this is the move
00:26:27.070
command at the very end he says sends another dr command that tells him to stop and checks the sensors again so i'm
00:26:34.090
going to and this particular demo the Rumba will actually move and he should
00:26:40.929
respond if he gets hit if he touches anything since I'm checking the center's
00:26:46.390
he should respond and stop moving I'm going to tell him to move a thousand
00:26:52.299
millimeters well now this is the other
00:26:57.580
interesting thing is that rumba if you pick him up at any time it when he's
00:27:04.000
moving if you pick him up off the floor or anything he instantly goes out of control mode and goes into passive mode
00:27:09.370
so what happened just now as i sent the command and told him to move and then there's all this sensor data that popped up on the on the monitor there and all
00:27:17.169
the sensor data was coming back and it was saying that there was nothing it was a formatted 0 in this basic sensor data
00:27:23.140
is just saying that it has not seen anything and that's the truth because room but not moved anywhere but you can
00:27:29.049
still get sensors from so what I want to do is I have to call sort of story tell
00:27:37.900
them that I want to take control again
00:27:43.080
and now now this time if I tell him to move he should actually move oh and in
00:27:51.669
Murphy strikes
00:27:58.880
okay let's try this again okay you can it's kind of our disabled on the monitor
00:28:05.130
their rumba the light is green on his clean button it's a little button in the middle and now that I took control his
00:28:12.029
light went off which means I'm in control okay so what you witnessed was
00:28:23.970
rumba moving I didn't get him to stop until I hit him a couple times and this
00:28:30.570
is actually part of the demo that was not a mess up and that's the pitfall the
00:28:36.059
next pitfall that you're going to run into with doing some physical competing or most anything type of physical computing is latency there's an issue
00:28:43.110
with working with rumba and physical computing is that if you ask him for
00:28:48.390
instance instead of are we there yet is have you seen anything yet have you hit anything yet you might ask him and then
00:28:53.730
over any type of wireless connection or deer because of the amount of time that room but has to take to read a sensor
00:28:59.820
and then send it back to you you might end up missing some data or you might
00:29:05.399
maybe you ask him the first time he hasn't responded so you just got Neil back you check again and then that time
00:29:10.860
your response and for instance I hit him three or four times but it was the last time where I kind of held the the front
00:29:17.669
bumper is when I actually got a response was positive that he had hit something
00:29:22.909
so regardless of the fact that there's a latency issue i want to just move on and
00:29:28.880
do something even I just want to cut out all them all the middleman and I want to be on a control room but even from a
00:29:35.970
server way out in the cloud and so now I'm going to move on and say let's get rid of the Arduino which we've already
00:29:41.669
done with the C code and let's get rid of the bluetooth adapter and just control the roomba wirelessly through
00:29:47.730
from some rails app and this is what how you're going to do it this is the device
00:29:52.830
called the Y fly this is after I've soldered some pins on to it and the whitefly is made me have to learn a
00:29:59.880
little bit about wiring and wiring diagrams and something called logic level converters for instance the Rumba
00:30:05.850
speaks on all those serial ports five volts that's his signals and while
00:30:11.669
the why fly itself runs on 3.3 volts which essentially means that if you were
00:30:16.739
to just connect those directly the wife line because it's very sensitive to differences and voltages would probably
00:30:23.489
just fry and you wouldn't actually get it to do anything for you these two particular device circuit boards on the
00:30:29.759
left right here is a power converter which allowed me to convert rumbas powers that way the wife'll I could be
00:30:34.889
powered directly by rumba and the one on the right is the logic level converter where I was able to it steps down the
00:30:40.980
Rumba voltage levels I've never done anything with this anything like this before and I've never done any type of
00:30:47.009
breadboarding I actually found that this was really really fun to breadboard although it doesn't really do it do much
00:30:52.769
other than just prototype and tell you what it should probably look like so how
00:30:57.899
do we actually control rumba from rail from a rails app or web app and up until
00:31:03.149
now we've been using a serial port gem to control it but actually since you're going to be connecting through why fly
00:31:08.909
over a wireless network you probably want to do it with an IP address and a poor and you can do that strictly
00:31:14.879
through using TCP socket which is default with any Ruby installer and so I
00:31:21.359
did everything I done all the psaltery and I set it up connected it and then I connected to my wife line with TCP TCP
00:31:29.909
socket and I was able to send Ruby's rumba a couple commands he moved but I
00:31:35.850
never got anything back from he wasn't speaking to me I couldn't get any data apparently the wife lion after which I
00:31:43.859
have which I'm not demoing here because I'm afraid wireless network might be a problem for demo but this particular
00:31:50.759
wife line is pretty cool because it can create its own ad hoc network as soon as you're connected to the room but it will
00:31:55.769
create its own network you can connect to the Wi-Fi and don't have to use a router but the problem with the ad hoc
00:32:02.399
network the universe that I had a problem trying to get any data back and
00:32:07.679
I probably pulled my hair out for like two days trying to figure this out and eventually I was googling trying to find
00:32:12.929
anyone to have this problem and I think there's only one other guy in the world that had this problem or even blogged about or said anything it was on a
00:32:18.989
Egyptian hackerspace blog and I found this guy that said hey I'm able to send stuff to my wife lie but I
00:32:26.440
can't get shipped back from it what the hell's going on and some dude replied like two days later and didn't even
00:32:31.809
answer the question which is cool and all but what really was awesome this guy came back even though they even though
00:32:37.990
he had already just solved the problem he said hey don't worry I solved it what it was is that in ad-hoc mode even
00:32:44.320
though in the documentation you can't receive anything and I was like oh god you're an angel thank you for write this
00:32:50.049
so after directly after that I dropped the ad hoc after I'd done the configuration settings from the wife'll
00:32:55.690
I connected it to my router and boom I was able to send and receive over the wireless network but the crazy thing was
00:33:01.690
as I started just getting a whole bunch of from Roomba like he used to sending me all this crap and I couldn't read it like it was not normal sensor
00:33:08.290
data and this is the number another pitfall and what you do when you're in this situation there's actually three
00:33:13.690
solutions and the first is read the manual this particular Wi-Fi
00:33:19.270
device I went back to the manual and I started reading all the options like what the hell did I do wrong why is he sending me a whole bunch of crap maybe
00:33:25.390
it's not from aruba and if that doesn't work reset your room by the factory defaults and the way you do this is pull
00:33:32.110
out the battery hold the power button down for a few minutes he'll and then put the battery back in he beats up all
00:33:37.179
lively and says oh who are you and then you have to strap another knife on him so that way he'll remember you later and
00:33:44.590
if that doesn't work which in my case is still did not work go back and read the manual again that's the third
00:33:50.260
solution and actually even better yet is a ask your coworker Thank You mouse for
00:33:55.630
reading the manual with you and then you end up finding this really obscure thing that you just assumed didn't really matter to you and for instance in this
00:34:03.309
thing the why fly by default like if you connect to it over a wireless network or over an ad-hoc network it doesn't send
00:34:09.010
you any debug information but by default this Wi-Fi was sending debugging information to the serial outs so what
00:34:15.220
he was doing is sending link he was pretty not everything the Wi-Fi as soon as I booted up would print out every bit of debugging information to the Roomba
00:34:21.220
and rumba was receiving all this data and just thought it was a bunch of commands and some of it would interpret and some of it would not interpret and
00:34:27.340
so apparently Roomba just got like flooded with thousands of commands like really quickly and was just sending me a
00:34:32.679
bunch of back that I could never figure out so the magic configuration with the Wi-Fi adapter is set your calm
00:34:38.780
close to zero say your combo pin' 20 and say your sis print level 20 now these three things say whenever you make a
00:34:44.630
connection to room to the vibe fly don't print anything on the comm port don't print anything on serial port when you
00:34:50.030
close your connection to the room but don't do anything and when you start up don't write any debug messages and that's it so after two days of
00:34:57.139
tearing my hair out that pretty prototype i started with ended up looking like this box of wires and
00:35:02.780
cabling that's a sad sight but that's what it was so this is a quick video of
00:35:11.290
yes it's funny isn't it and this is me controlling my Roomba wirelessly over
00:35:17.450
the network sorry it's such a bad video but it's kind of hard to see I've just called roomba new and I send an IP
00:35:23.810
address it was the 192 local IP address the light on the room but went off which means he's under my control and then I
00:35:30.910
send him a move command and I just hit it and there's a little bit of latency
00:35:36.740
about a second and a half and then Roomba is spinning happily in place
00:35:41.870
where the box of wires not too cool but it it was fun that's my Roomba and the
00:35:48.380
white flag so now I'm going now that I've talked about why fly and wirelessly connecting I want to talk about three
00:35:54.980
final hurdles and the three final hurdles have nothing to do with dogs on rumbas nothing to do with cats on rumbas
00:36:01.430
and nothing to do with bird bears playing keyboards actually the three
00:36:06.470
final hurdles have to do with why fly and and using rails apps and I'm going to get to the rails demonstration in
00:36:12.620
just a moment but whenever you now that we've got all the base code how do you actually talk to the roomba using rails
00:36:18.830
and what you would normally do is if a request like you make a request through your web browser to some rails app code
00:36:24.620
and it would open up a TCP socket to your room but it would pass a command to the roomba overthrew the Y fly why fly
00:36:31.310
would send the command to rumba rumba would reply with like for instance if you told them to give me such move and to give me sensor data keep giving me a
00:36:37.070
sensor data what you have to do is hold that socket open and just keep reading now the problem with that is you're
00:36:42.650
holding and you're holding and you're holding and your request doesn't finish so the browser won't actually be
00:36:47.990
to display what's happened until after all that request has happened so what happens if I tell room but to move to
00:36:53.420
miles I'm going to be holding for a very long time while Roomba is moving for two miles now it works fine if you're just
00:36:59.180
telling remember to move like 100 millimeters because that's less than a second so the request finished is pretty quick well now this is the cool thing
00:37:05.600
about the Y fly they actually the actual piece of device the white flag is it has another option that you can configure it
00:37:10.820
with where for instance if you take a rails app and you make a request over TCP socket to the room button you tell
00:37:16.130
room but I want you to move and I want you to update me with sensors every 15 milliseconds the Wi-Fi has this cool
00:37:22.070
option where it can if you configure it take any data that it receives over the
00:37:27.590
serial port and automatically forward that in some restful manner to any web server on the planet so you could just
00:37:33.740
say here's the URL that I want you to poet to post or to send any of this data any of this data you get so we just post
00:37:39.320
all those bites that it got back and it could do that every 15 milliseconds perhaps so that's that's a cool way to
00:37:44.540
get over that hurdle another roller that you have is the stateless web text where every time you're making a new request
00:37:51.320
the room but you have to you have to instant it initialize the new roomba object and rumbas has to sleep we have
00:37:57.619
to wait a two-tenths of a second plus another tenth of a second to start the room but because the Roomba has to be
00:38:03.680
able to have time to boot up before you start sending the commands that's another hurdle and the third hurdle which is very similar is a device busy
00:38:09.890
if if you have two concurrent requests that happen at the same time from rails app and the first one gets throwing these has the port open to the roomba
00:38:16.100
the room but can only service one master at a time the other request is just going to get an error busy and you won't
00:38:21.710
be able to get anything so that's kind of broken Oh quick I try to hack up some way to fix this these problems and I
00:38:29.119
came up with this really weird idea that maybe I could have like a socket server that runs maybe the first rails request
00:38:35.119
that comes in we'll start this socket server and this is some pseudo code on the right and the socket server would
00:38:40.160
create the create an instance of the roomba class and hold that instance open
00:38:46.460
for all future requests until maybe it timed out at some particular point and that way you could every request that
00:38:52.340
comes in immediately afterwards would not have any boot up time wouldn't have to wait at all for a room but to sleep
00:38:58.369
or get any commands to get set up so you could just pass many and also the socket server could be like a queue so if you send them a whole
00:39:03.880
bunch of commands really fast it would queue up and it wouldn't cause room but to go crazy so now I'm actually going to
00:39:10.210
do an extended demo with the rails app which is probably the one thing that you guys were wondering where is the rails in this and let's see if I can get this
00:39:23.850
okay so i've got a rails app that I've written quick move try to get photobooth
00:39:34.570
on here too
00:39:41.170
okay I don't know how I'm going to do this very well but his video here's the
00:39:50.349
rails app it's kind of hard to see but I've listed I've just created so if you were to go in here and this is rumba
00:39:57.010
blue because I'm running on the Bluetooth adapter you could edit Gribble blue and essentially all it's doing is
00:40:02.650
the location where is remember blue and this is a serial port then in normal cases if you're using y flye you would actually have an IP address with a port
00:40:10.000
on the IP rubra blue here when you come in you can either take control room a blue or you could exterminate him if
00:40:16.750
he's a Dalek Doctor Who fans I heard some people laugh okay so this is a
00:40:23.380
Roomba and i'm just going to take control and if we're lucky in about three seconds a report command goes and
00:40:30.670
boom this is some information i'm pulling from roomba sensors it's telling me right now sorry guys this was hard to
00:40:37.569
see tell me right now that Roomba is in safe mode his battery charge is 2500
00:40:45.329
milliamp hours out of his total on this battery the current amount of milliamps
00:40:50.920
that are running out of room buzz- 211 and get his temperature of his battery you can find out if he's dr. undocked is
00:40:58.660
he charging etc and I've just got this running on ajax it's kind of periodically pulling rumba and saying
00:41:04.299
hey give me a report on your sensors and now what we can do if i get this camera for you guys to see i can tell room but
00:41:11.079
hey i want you to move 100 maybe i want you to move 500 millimeters which is not
00:41:18.700
really that far and i just want you to move at the default velocity of 200 and
00:41:24.010
some lot and rumba moves and it sends back all the sensor data here these are bumps and drops and telling you if you
00:41:30.130
hit anything we could also tell them hey i want you to move a little faster i want you to move back backwards and a negative velocity five hundred and i
00:41:36.579
want you to move you know once you move
00:41:41.890
back a thousand millimeters so okay rumba let's go backwards I remember it
00:41:48.220
goes a little faster and as additional probably the most
00:41:55.100
practical thing now this is a way that you could control Roomba manually if you wanted to tell him to move probably one
00:42:00.230
of the more practical things that you would want to do with a rumbo remotely over the web would be to set up a schedule because scheduling him by
00:42:05.660
pushing the buttons down there as a pain in the ass you have to actually bend over and start pushing buttons what the hell is that for instance you could just
00:42:12.310
you could just send him hours and minutes of what every day of the week that you'd want to set the schedule so
00:42:18.770
for instance you'd tell him that on mondays i want you to clean at ten-thirty and he would say yes my lord and that would be schedule updated so
00:42:26.150
now if I go down there and i would if i push the button to turn them on this check his schedule and actually do the manual thing it would show up as 1030
00:42:32.210
it's already updated now you could also clear the schedule so this would be really cool if you're out on vacation
00:42:37.760
you're like oh I'm going to be home tomorrow let's clean the house remember go clean the house right now and you log
00:42:43.580
in with your iPhone there's some other things like the motors you can control
00:42:49.250
all the motors I'm not going to go through this demo because I think I'm running out of time another thing that
00:42:55.190
you can do I don't know if you guys can hear this but Roomba has a midi like he can play midi notes so you can queue up
00:43:02.180
songs and then tell him to play these songs for instance i'm just going to tell him here's all the notes though these these like map out to like
00:43:09.340
particular notes on a keyboard or whatever in so maybe some of you heard
00:43:16.700
that maybe not a whole lot of people and now i'm just going to go into like a full demo and if everyone's quiet there
00:43:22.910
might be a surprise in this demo rumba rumba is going to try to sing us a really cool song while he's running around this full demo which all the
00:43:29.990
sensors up here would be updated hopefully we'll run through a whole
00:43:35.090
bunch of commands and i'll try to tell you what's happening while it's going on
00:43:47.109
so room is just turning around proving that we can control him to tell him to move in a particular direction and
00:43:53.769
tongue back up and just turn out particular any particular voters now I
00:44:00.440
tell them turn on the water I don't you spin I'm going to tell them reversing goal of the faster pace pace so these
00:44:06.140
are just basically commands telling the singing yet
00:44:14.980
I can tell them to go into a regular cleaning mode course not and then I tell them to turn off and I turn it up into
00:44:24.340
two different cleaning modes and then I turn them off because I was afraid that you might go a little like a little bit too far away from the stage but that was
00:44:32.020
me controlling remote from the rail south thank
00:44:42.890
so looking forward I want to oh cool I got a couple minutes left looking
00:44:48.230
forward I just want to talk about where I'm at now and what out what I wanted to do and one of the things when doing with
00:44:53.839
physical computing devices is how the hell do you test them how do you write test to test something that's in the
00:44:59.630
real world as opposed to just software there you can easily test and I had an interesting story told to me that when a
00:45:06.859
guy was talking to a guy from north of grooming and particularly asked this question like how do you test those
00:45:11.990
drones that fly around and stuff and the guy said what do you mean and he's like well how do you write like software test
00:45:18.410
for these and he said well you write a simulation and then you test the simulation is ok you test a simulation that's great and then how do you know
00:45:24.079
that that matches the real thing you say well you really don't you just have to keep testing the simulation and
00:45:29.539
comparing it to the real thing to see if it's the same and so my idea was a couple months ago was it would be really
00:45:35.119
cool for all those people that don't have rumbas if I could write like a software simulator a software simulation of rumba and that way we could write a
00:45:41.390
lot of really cool scenarios and test those scenarios against the software and then I could run that same scenario on
00:45:48.200
the Roomba and see if given the same arbitrary boundaries of some particular distance and some obstacles in there
00:45:54.289
would he return the same sensors at the same time as the simulation did and so I set out and tried to make a quick simulator I wrote a simulator and then
00:46:01.880
we ran tests and scenarios against the simulator and then we ran some of those simulations with the real thing this
00:46:07.160
simulator it's in the project but it's not it's not up to par it still has a lot of work to be done but if you're
00:46:13.400
just moving forward and backward and some really basic moves it it's pretty close so that's a pretty cool but doing
00:46:19.160
doing a lot of the making Roomba move at angles and stuff that's hasn't been very
00:46:24.680
easy to make so it'd be really cool if you could just write a whole bunch of scenarios anyone in the world and say
00:46:31.430
hey what would happen if this physical device done this and then BOOM roomba of course would say bring it I'm ready and
00:46:38.690
then you could maybe queue up all these simulators and then control the control room but over the web that would be pretty cool for instance what if you I
00:46:45.170
taped a marker on the side of rumbo here would be a pretty cool challenge could any of you draw something on the floor if I tape it mark around room but could
00:46:51.650
you like simulate him moving around to draw like your name on a piece of paper if you had are on them and then if you were to run
00:46:57.920
that in a simulation and run it in real life how close with the real-life drawing be to the simulation where could
00:47:03.439
you map out the room or something more complex and so physical computing so why
00:47:08.959
the hell should you care why should you care about rumba and rails etc and it's not just because techno viking tells you
00:47:14.900
you need to care but i'm going to give you at least 50 billion in one may be reasons by 2020 there are going to be 50
00:47:23.390
billion connected devices from an ericsson white paper and TechCrunch had a recent published article that talked
00:47:31.670
about 2012 that the number of mobile wireless devices already exceeded the population out of those mobile devices
00:47:39.439
by 2016 there's there's going to be 16 billion mobile devices and a large percentage of those are going to be
00:47:45.739
machined to machine only so there's going to be room for software they only controls machines it's not just
00:47:51.079
particularly for your mobile device like your iPhone so maybe people will want to automatic or control room but over the
00:47:57.229
web for instance everything that was Consumer Electronics is going to be moved into just about everything is
00:48:02.599
going to be networked it's going to be ubiquitous computing right some people are doing some critic pretty cool things with Wi-Fi I'd like to talk about those
00:48:08.150
but I think I'm running out of time this one for instance was just a quick proof
00:48:13.789
of concept that a guy made a clock with a why fly and on this clock he gets updates from Twitter and the weather
00:48:19.819
report and all sorts of things that the why fly is being pulled from servers and display this guy done a really cool
00:48:26.150
project in New Zealand where he used an Arduino and some water sensors and he
00:48:32.509
was able to put water sensors and tanks around his giant farm and wirelessly over the web he was able to pull those
00:48:38.779
and find out much water was in his tank that seemed like a pretty practical thing to do if you're not looking for
00:48:43.969
all that practical are just looking to really get into robotics and you but you don't really want to Ruby you could get something like a create which I robot
00:48:50.119
cells is the the machine without all the vacuuming parts they're pretty cheap easy to get and maybe you can make this
00:48:55.999
Bionic hamster version that they got here where a hamster is running around in the wall on a wheel and the sensor
00:49:01.249
underneath it picks up and knows to tell remember to go that direction that seemed pretty cool
00:49:06.890
uh so I want to finish my last reason why you should care about hacking rumbas
00:49:13.589
is more to do with what I've talked about in the beginning that the greatest obstacle to discovery is not ignorance
00:49:19.890
but the illusion of knowledge and in this sense if you're ignorant of physical computing I think you should
00:49:25.650
adopt the pragmatic philosophy and just go out and try it see what it's like invest in your knowledge portfolio and
00:49:32.670
in fact I think you should be like a guy named Simone Stevan and if any of you guys have ever heard of Simone's Devon a
00:49:38.869
historical figure and he's in he was an eccentric this guy was in the 1500s he
00:49:44.460
was pretty crazy because he was one of the only people they had amphibious vehicles he drove this thing around is
00:49:49.740
called a land yacht I would pretty I would be sure to think that he was kind of weird if I saw him in this but so
00:49:55.619
most everyone was pretty kriti interesting because he worked on so many different things throughout his life and
00:50:00.839
most of the things had nothing to do with his forte which was just engineering of bridges and dams and
00:50:06.420
stuff but in 1585 he published this book called the tents which was hey I want I
00:50:11.490
think we should use the decimal system for for all your measurements and stuff
00:50:16.560
at that time most people were just using fractions like one-third they were right one-third as opposed to like point three
00:50:22.140
three etc and this small thing that he did actually ended up he just published
00:50:27.810
it it was one of the small things he didn't think it was all that important he'd never really dealt with publishing about measurements ends up changing the
00:50:34.950
world where everyone uses the decimal system and what he ends up saying is what seems a wonder is not really a wonder and if you are ever like thinking
00:50:43.680
that some of these physical computing projects are just too difficult to get into take it from me if it's if it seems
00:50:49.410
like a wonder it's not really wondering it can be done so if you have a chance for kit and thanks for coming
00:51:27.420
I
00:51:47.200
member