RailsConf 2022 - Opening Keynote: The Journey to Zeitwerk by Xavier Noria

Opening Keynote: The Journey to Zeitwerk by Xavier Noria

RailsConf 2022

00:00:00.900 foreign
00:00:12.080 and now for our opening keynote speaker noria comes to us from Barcelona he's
00:00:19.859 been on the rails core team since 2011. he's a part-time lecturer lifelong
00:00:25.560 learner and Ruby hero Award winner he's done tons of rails co-work on zeit work
00:00:31.439 among other things please help me give a warm welcome to Xavier
00:00:42.980 uh first of all let me thank uh reliscom for invited me to give discount this
00:00:49.800 keynote which is truly an honor for me [Applause]
00:01:02.100 all right auto loading so for those those of you new to rails
00:01:09.860 autoloading is that feature in the framework that allows you to just
00:01:15.360 program having all your application classes and modules just available
00:01:20.580 everywhere without using require this is also related to uh reloading and
00:01:28.200 eager loading okay but that's that's autoloading in the
00:01:33.900 rails framework Auto learning has been in rails since the beginning
00:01:41.780 and in Rage 6.0 we ship it a new implementation an alternative
00:01:47.640 implementation of auto loading which was coexisting with the previous autoloader
00:01:54.720 for 6.0 for 6.1 and that was like a transition period that took us to rail 7
00:02:03.479 where we finally dropped the previous implementation because the new one is better
00:02:09.360 this is the commit where this happens this is my commit dropping the support
00:02:15.000 for the previous autoloader which we we called the classic autoloader all right
00:02:21.900 this commit is very symbolic it's very symbolic because there's years behind
00:02:28.200 that commit and this was like a entire Journey
00:02:34.739 reaching this point that I would like to share with you today
00:02:40.500 all right so Constance even more back in the day is a not very
00:02:49.200 well documented feature of Ruby and back in the day
00:02:54.379 rails didn't have even documented auto loading so
00:03:00.319 years ago I was interested in these topics and did a research by myself
00:03:06.540 trying to reverse engineer how things work and as a result of that research at
00:03:11.940 least I could share some of the things that I have I had learned with talks for
00:03:18.000 instance and with the auto loading guide this autoloading guide had one Infamous
00:03:27.060 section which is the section about gochas in the Classic Auto loader here's
00:03:33.420 a scrolling screenshot of that section [Applause] that is so long that I had to put it
00:03:39.780 diagonally you know in the slide
00:03:46.340 why why did this happen well the technique the Classic Auto
00:03:53.040 loader uses so it's based on has fundamental limitations so this section
00:03:59.580 was like documentation I'm sorry this is the way it works it is not technically a
00:04:05.340 bug because uh the coaches were a consequence of the limitations of a
00:04:10.980 technique itself so I would like to leverage this presentation to explain to you why
00:04:20.160 did we have did this section with gochas in the previous autoloading guide
00:04:26.100 so let's do a quick very quick recap about the most important things we have
00:04:31.139 to know about constants in Ruby the first thing that we have to know is
00:04:37.380 that constants belong literally to class and module objects this is something
00:04:43.220 quite particular in the Ruby programming language because in most languages a constant more or less is just like a you
00:04:51.180 could say it's like a variable whose value is protected against changes something like that you know it's like
00:04:58.080 very simple topic but in Ruby it's a very deep topic and this is one of the particularities
00:05:05.040 of the Ruby programming language constants is something that belongs explicitly to class and module objects
00:05:11.940 so you have to imagine that every single class a module object has like an
00:05:17.340 internal hash table that Maps constant names constant names to values
00:05:25.139 that belongs to the Ruby model all right and you have API to manipulate that hash table quote
00:05:33.840 quote which is the constants API you can ask a module or class object about the
00:05:39.840 constants it is storing you can dynamically set and remove constants there's an API around this concept okay
00:05:48.000 top level constants belong to object so if you say 4 equals 1 at the top level
00:05:54.300 in a ruby script that full constant capital letters is stored in object
00:06:01.259 object receives a new entry in that hash table is not just like a pure identifier
00:06:06.960 like a variable okay it it has more consequences it is stored in the
00:06:12.960 internal High stable code in object all right
00:06:19.440 so we have constants stored everywhere and when you refer to a constant in
00:06:26.460 source code Ruby has to do a lookup to find that constant we're going to see
00:06:32.580 two algorithms that perform lookup in different ways the first one is the algorithm that is
00:06:40.440 being used when the constant reference is relative okay a relative constant reference is like the regular one the
00:06:47.280 one does not have column column to the left so for instance in this source code
00:06:52.740 we have admin users controller and an action new and that action has
00:07:00.300 so that user the user constant because Ruby does not have Syntax for types
00:07:06.840 so when you see this user here this is not a type like in other programming languages this is a regular constant
00:07:12.780 this is an expression that evaluates to a class object all right so this is a relative constant
00:07:18.960 uh how does Ruby resolve this constant
00:07:24.000 there's a lookup going on let me first introduce to explain this the concept of nesting okay
00:07:30.060 so at any given point in the source code of a ruby program there's something that The Interpreter
00:07:37.740 is is keeping internally for you call it nesting which is a collection that
00:07:43.199 reflects the nested classes and modules that you have at that specific point in your
00:07:49.680 source code so we have here two examples with two different nestings okay in the first one we have module a
00:07:57.120 Class B and then uh in the spot where we have the comment
00:08:04.080 the nesting is reflecting this nested modulant glass so it's a colon column B
00:08:11.460 a okay now
00:08:16.560 in the second example we have class a colon column B directly
00:08:22.199 so the nesting is different why because you see there's no module a
00:08:28.440 and then nested a class keyword you only have one keyword class
00:08:33.899 so the nesting only gets one plus object pushed all right so a is not in the
00:08:40.740 nesting in the second example okay
00:08:46.860 enough to understand rosly are you not going to be very detailed but enough for
00:08:51.959 for this talk how this user constant is look it up
00:08:57.120 so you could think why uh user is like the top level okay it's like the global
00:09:04.019 user right this should be immediate uh it is not there's there's a lookup
00:09:11.480 and from this spot in the source code to the point where Ruby finds the user
00:09:18.779 constant does a number of steps so it goes like this
00:09:24.779 we saw what's the nesting so the first thing that Ruby checks is the nesting
00:09:30.899 because for Ruby this is the way to emulate the concept of a namespace
00:09:37.560 so we have to think that we are in in we are within a namespace there
00:09:45.060 that has to be checked first then if within that namespace we do not
00:09:50.640 find things then we continue looking up the constant so it goes like this
00:09:56.100 the user constant does the user constant belong to the
00:10:01.980 admin column column user's controller class let's suppose the answer is no
00:10:07.380 let's suppose we are looking for the top level one okay but first that one is check this is the
00:10:13.440 name spacing emulation of Ruby now it is not found there okay
00:10:20.339 then we still check the nesting upwards
00:10:25.380 admin do you have a user constant no all right let's continue the search the
00:10:31.200 search now continues we have exhausted the nesting then the search continues in the
00:10:37.560 ancestor chain upwards so the superclass of admin column column users controller
00:10:42.600 is application controller application controller do you have user no
00:10:48.600 goes upwards upwards eventually object is in the ancestor chain of all classes
00:10:55.079 So eventually object is going to be check and remember top level classes are
00:11:01.140 stored in object So eventually you hit object and then
00:11:06.600 group is going to find the user constant and then you get the value and the
00:11:12.120 program resumes so you see there's a number of steps
00:11:17.399 if we made a typo or something and the constant really was not defined
00:11:23.339 in the end there's a fallback which is a callback called cons missing so if the
00:11:30.720 if the lookup failed cons missing is going to be called and all classes and
00:11:37.079 modules respond to cons missing the default implementation races name error but you can overwrite that
00:11:43.380 implementation and the Classic Auto loader did that as we are going to see
00:11:49.800 now absolute constant lookup this is easier all right so in the
00:11:56.579 previous one in the previous one we did first the nesting AdWords and then the
00:12:04.140 ancestor chain um upwards and there's a fallback two
00:12:09.600 cons missing okay this one particular exception with
00:12:14.820 modules because modules do not have object in the ancestors so if the inner
00:12:20.100 namespace is a module then Ruby manually checks object to a still find user okay
00:12:27.480 well so this is the main idea first the nesting outwards then the ancestral
00:12:34.140 chain upwards and fall back to cons missing now
00:12:40.019 here we have product first is a constant
00:12:45.240 and product is this one is resolved with the previous
00:12:50.420 algorithm because that one is relative it doesn't have column column to the left so first Ruby
00:12:57.440 finds product okay let's imagine that everything is correct and product gives you a class or module object then
00:13:05.779 update eventually job is the one that is going to be resolved with the new
00:13:11.220 algorithm that we are going to explain that one is like qualified absolute okay it's a scope
00:13:17.639 scope to product so what happens here basically you check in product if it's
00:13:24.060 not in product you check the ancestors okay and if the ancestors had
00:13:29.760 object nowadays object is skipped because otherwise you could have product
00:13:35.220 product column column sorry product column column string for instance and it
00:13:40.320 could be found that's strange because a string is top level is not let's go so
00:13:47.040 you check in product and you check the ancestor chain of product and a skip object if present
00:13:53.639 easier all right so now we have the knowledge to
00:14:00.120 understand why the Classic Auto loader couldn't work
00:14:06.839 because what was the idea behind the Classic Auto loader so we have the final conventions that you know all right so
00:14:13.320 you know that in rails the file names match constant paths with
00:14:20.279 namespaces are playing the being represented as directories right
00:14:25.920 we have the file name conventions then we have the auto load paths the autolo
00:14:32.760 paths are app controllers app models app helpers okay those folders in the
00:14:39.300 project that represent the top level namespace opt and here's the thing rails defines a
00:14:46.980 cons missing hook in the Classic Auto loader and that consummation hook
00:14:53.220 there's a few hundred lines of code there but the the raw idea the basic idea is the the following if you get the
00:15:00.420 user constant that is missing then you get the name in cons missing
00:15:06.300 hey this constant is missing then the autoloader underscore that cons that
00:15:11.579 constant name to get user lowercase and basically Walcott the auto load
00:15:18.180 paths looking for user.rv that's the idea you get called you underscore you
00:15:24.360 look for the file you find the file the file you load the file everything works okay so make sense
00:15:32.220 however the problem is that cons missing does not get the nesting
00:15:38.940 cons missing does not know if the missing reference was relative or was
00:15:44.339 qualified so cons missing and the Classic Auto loader basically cannot emulate the way
00:15:52.199 Ruby resolve constants does not have the necessary information that's like fundamental
00:15:59.060 intrinsic limitation of this technique however even if you had that information
00:16:04.860 there's still one gocha which is that module comes missing as we have seen in
00:16:10.980 the algorithms is the last step so if you want to outload something
00:16:19.019 and in this lookup you find a constant with the same name
00:16:25.139 elsewhere Ruby is going to be able to resolve it not in the place that you were expecting
00:16:32.579 but Ruby could resolve that constant in another name space
00:16:38.759 what happens then that comes missing is not even called
00:16:43.860 because it's the last step if you do not hit the fallback
00:16:49.079 bad luck I could resolve the constant so classic just couldn't so classic was
00:16:56.040 extremely helpful it allowed us to work without requires for 15 years
00:17:04.100 but unfortunately it couldn't match the way Ruby works with constants
00:17:12.240 now there's another thing in Ruby call it auto load all right
00:17:20.160 autoload is a method that allows you to say in class and modulus okay
00:17:25.559 if you are looking for the user constant in this example please load the file
00:17:31.440 that goes in the second argument okay
00:17:37.140 and the key Point here is that when the constant lookup is happening
00:17:44.400 and Ruby is checking class and module objects do you have this constant do you have this constant
00:17:50.100 the logic is a little bit deeper because
00:17:55.260 what happens in reality is that if the cluster module does not have the
00:18:01.260 constant but has a now to load Define it for that constant
00:18:07.500 then it executes out a lot so this is this is a key observation
00:18:14.760 that I realized it like years ago and that tells us that if we could Base auto
00:18:22.260 loading on this feature it could just work because that's built in in The
00:18:27.840 Interpreter that is built in in the lookup paths and it's going to respect everything about the semantics of Ruby
00:18:33.539 and constants because it is Ruby doing the job at the right place at the right time
00:18:41.600 so there was like a rose idea of one possibly alternative way to implement auto
00:18:48.720 loading which is the following you still have to find them conventions you still have the auto load paths
00:18:56.760 but then instead of reacting to cons missing you go beforehand you scan the project tree
00:19:05.460 you first scan before the application is being execute let's say first scan
00:19:11.400 and if you find user daughter B in the file system you caramelize that
00:19:18.240 and set you define an auto lot for user that is going to load that particular
00:19:24.419 file that was the idea I have had this idea for a long time in
00:19:33.179 making this happen watch for me like a like a Grail project
00:19:39.000 however there were some technical difficulties implementing this
00:19:45.179 let me show you them there are a number of them
00:19:50.340 and then there's a mystery one at the bottom that is going to be a secret for a little bit
00:19:58.260 so let's go one by one okay the first one is that module autoload
00:20:05.280 used an internal require so in this alternative implementation
00:20:12.179 you want to be aware of when things are being Auto loaded because you want to
00:20:19.500 keep track of what is being Auto loaded maybe you want to lock a trace maybe you
00:20:24.539 want to trigger callbacks you know you need some housekeeping in order to do that
00:20:30.179 Auto load loads things with require
00:20:35.520 the technique is to decorate a thin thin wrapper around require
00:20:41.520 but but Auto load was not using the kernel
00:20:46.860 required that we know was using an internal C function that was not
00:20:52.200 reachable from Ruby so ah we cannot we cannot decorate
00:20:58.380 require so we have a blocker there unblock this this fine gentleman
00:21:06.990 [Applause]
00:21:12.960 this fine gentleman Aaron Patterson so Aaron patched in this commit
00:21:19.980 Auto load so that it went through cannot require the motivation for this
00:21:25.260 particular commit was that ruby gems also decorates kernel require so regular
00:21:31.919 requires go through ruby gems and it has control over them but requires trigger it by Auto load whereby passing that
00:21:39.240 decoration so with this Commit This got Unified this was in 2015 as you can see and
00:21:46.620 shipped with Ruby 2.3 increase Mass 2015. so
00:21:52.740 hey this is solved now let's go for another one
00:21:58.679 requires ident as you know if you require something and then execute the the same required
00:22:06.600 this is an artificial example but you know the the common use cases that you have requires for the files in different
00:22:12.840 places in your project so only one of them is going to be executed and the
00:22:18.360 rest does not need to be executed and require this item this is a problem for reloading
00:22:25.559 because since autoload users require if you reload as we do in in rails and
00:22:32.340 we want to reset the state of the project what happens okay we have to scan again
00:22:39.500 set auto loads and when the auto load is going to be triggered I'm sorry but the
00:22:45.360 required is not going to fly why because or it was already executed in the previous before the reload
00:22:52.200 all right so uh this prevented the idea of reloading
00:23:00.659 now before doing Ruby I did pal for some years and impel you do quite dirty
00:23:07.140 things and there was an analogous situation for
00:23:12.720 which I knew a hack how does require know that the file was already loaded it
00:23:19.799 keeps internally um a collection of files that have been required call it loaded features
00:23:26.940 so loaded features is mutable and if you [Applause]
00:23:36.000 if you've removed the file you get the required working again
00:23:45.539 all right so we have one way to make reloading work
00:23:52.020 yes okay we have two now
00:23:57.179 there's no API to remove an autoload so we have seen the auto load call that
00:24:03.059 allows you to set an autoload but if you reload you had set an auto load for something
00:24:09.120 and then the user delete that file to be clean
00:24:14.340 and the project should have no knowledge of that file you cannot you cannot you know carry that auto load forward
00:24:21.179 because it doesn't make sense for the current state of the project after the reload so to be clean
00:24:27.500 ideally you should just remove that auto load but there's no API for that
00:24:34.080 so Matthew Draper is a member of the Rails core team and
00:24:39.480 we shared with Matthew this bu for this alternative implementation and we had
00:24:45.179 some conversation with Matthew along the time that we're very encouraging and inspiring like yeah I don't know you
00:24:51.900 know checking options thinking about the problem all right and I learned from
00:24:57.000 from Matthew the way to overcome this one which is that it happens that if you
00:25:04.020 remove const as if the constant was already loaded the auto load is gone okay
00:25:10.440 so this makes sense because uh then I learned it with time that the the way
00:25:17.760 Ruby understands Auto loads thus in the API does not quite distinguish constants
00:25:24.480 that have been like really loaded from those that have an auto load set so this is consistent with this that
00:25:31.559 view of the API and it works you just remove cons the auto lot is gone
00:25:37.200 right closer no what about implicit name spaces
00:25:43.440 implicitly implicit name space is an image space that does not have a ruby file defining okay so you may recall
00:25:51.539 that if you have for instance an admin name space if you do not need to define something
00:25:58.980 in the admin modular class you do not need to write an admin file so rails
00:26:04.880 sees that those directories call it admin that's enough grades is going to
00:26:10.279 define a module for you call it admin okay
00:26:15.900 in the previous implementation well you get called admin okay I Define it on the
00:26:21.720 Fly resume but here you are not called you have to be proactive okay
00:26:27.960 so I didn't know how to do this but there was like a couple of ideas one idea is
00:26:34.140 well if I hit a directory at that point at the scan time
00:26:39.299 Define the module and continue recourse okay this has the cons that is not as lazy as
00:26:48.179 the previous autoloader it's not as lazy it it could be a good enough compromise
00:26:53.340 I believe but well there's another tentative idea which is to set an autoload on the
00:26:59.580 directory that would mean that there's going to be a require on a directory
00:27:04.820 this is a little bit strange because requires supposed to look for files
00:27:10.500 but well it was a tentative idea so this was not like I don't know which is the solution to this but I believe this can
00:27:17.100 be solved somehow you know and then we had this last
00:27:22.620 blocker explicit name spaces let me tell you what's an explicit name space you
00:27:27.900 have the name is space Define it in a file so
00:27:33.299 in hotel we you have Hotel RV and hotel pricing okay uh
00:27:39.360 in order to evaluate Hotel daughter B we have to have Hotel column column
00:27:45.120 pricing Define it because is there at the top level with an include at the same time
00:27:52.080 we cannot evaluate Hotel slash pricing that will be because that needs hotel to
00:27:57.960 be defined so it's like a chicken and egg problem we cannot go for any of the two
00:28:03.480 what to do with this so I had no idea what to do with this but I started working on that
00:28:11.220 all right this is uh for curiosity the message that I I put in the in in our chat in the core
00:28:18.659 team announcing that I I don't have all the answers but I am going to sit down
00:28:24.360 and try to solve this problem no
00:28:29.940 the next person to come here is my friend Rafael Francis from the race core team
00:28:35.340 that says hey uh we solved this in a in
00:28:41.159 in a hackathon in Shopify what
00:28:48.720 so this this conversation is so important that I'm going to share with you all
00:28:54.179 right so I said the next day I announce this I say got a prototype of the auto loading
00:29:02.220 idea working without name spaces because name spaces is what complicates things
00:29:07.740 includes reload and it is about 40 lines of code name spaces are what the fun is
00:29:13.679 but the boob strapping feels good so we got the ball rolling okay
00:29:19.260 and then Raphael says hold that thought I'll open source something
00:29:25.559 there you go with a repo and I say that's awesome I am in bed now can check
00:29:31.260 the code how do you define out a lot for no no no no Anonymous name spaces so I
00:29:36.960 am in bed now can check the code you who are you fooling to I of course wake up and went to
00:29:43.380 to find out immediately so the solution was Trace point
00:29:49.500 and Dylan soccer Smith was the outdoor of this proof of concept let's say in
00:29:56.279 Shopify and I I have never met him but I own this guy an Infinity stream of beer
00:30:03.179 or scotch or water or whatever he fancies because this was the the
00:30:09.120 inflection point for this project so this is the idea trench Point allows
00:30:15.179 you to to Define callbacks for certain events that happens when the program is
00:30:20.940 executed there's one callback which is call me when a class or module is
00:30:26.039 defined so if you set a callback for this event then you can do the
00:30:32.820 following when the class hotel when the class keyword is executed and the class
00:30:38.760 and the hotel class is defined before you reach the next line you will
00:30:44.460 call it so the hotel class is defined but include pricing is still not
00:30:49.860 executed so you are on time to go to the directories that conform
00:30:55.980 the hotel limit space set auto loads for hotel and continue
00:31:03.419 yes however is this performance because Trace point you know has a reputation of
00:31:09.480 okay if when I when I when I running Trace points uh it has overhead so does
00:31:16.500 this has an overhead I reach it to Richard Niemann
00:31:22.080 um from Heroku called triage he check it
00:31:27.380 some projects of him having the trace Point Define it
00:31:35.760 to see whether by itself created overhead he didn't find any any measurable
00:31:42.240 difference I also reached out to some saffron so he checked things out
00:31:48.419 uh there were some feedback that I addressed but the conclusion was
00:31:55.860 this seems to be fine we in the benchmarks we don't find anything that is measurable
00:32:02.580 so we have it we have it except for the secret but we have it
00:32:09.779 so at this point I knew we had it no you you have to sit down and write it
00:32:15.840 because from the growth idea to an actual implementation that can run in production there's a myriad of micro
00:32:22.320 decisions edge cases and things that you have to work on but we have it now you have to sit down and do it
00:32:29.279 so I sit down this was a very intense period for me I am a consultant I even
00:32:36.779 build it less to work with this because I was really into it
00:32:41.880 and I started working in active support because the previous Auto loader was
00:32:46.919 defined in active support okay the previous Auto is implemented in one
00:32:54.240 module basically the main parts of the logic call it active support column column
00:33:01.140 dependencies this module has model functions
00:33:06.240 that are like excuse me that are managing Global State Global State I
00:33:12.899 mean so which constants have been autoloaded which are the auto load paths all those are things that are one per
00:33:20.100 process you know and that's the way it works so I started
00:33:25.799 doing something similar only with the New Logic right so
00:33:32.880 when working on that I thought why I am defining module functions that
00:33:40.580 manage Global State this could be a class that you can instantiate
00:33:46.799 and has its own the object is the object has its own state
00:33:52.919 well yes I I started doing that refactor just for the sake of you know of
00:33:58.740 programming but when I was working on that I made a connection with another
00:34:05.760 pinpoint of mine that pain point was that in any
00:34:11.419 regular Ruby project that is not like small in my experience getting the requires
00:34:18.359 right is difficult it's difficult because require has Global side effects
00:34:24.240 and then you forget them you know and then depending on the cold paths you realize that at some point you forget to
00:34:31.379 require something but maybe that's a Heisenberg thing because depending on how you've reach that point maybe it was
00:34:38.280 loaded before it's brittle also how do you organize the required so that you can load one particular file
00:34:45.300 inside an image space and have it self-contained it that's difficult
00:34:51.419 so I I had a realization if we have different instance
00:34:59.180 managing different Source trees we could have loaders coexisting in this
00:35:05.040 in the same process managing different projects so then I changed the scope of the
00:35:11.640 project so from working on Rails we switch to
00:35:18.540 wanted to have an independent gem design it for any Ruby project
00:35:25.500 supporting coexisting Auto loaders in the same process and then the idea is we are going to do
00:35:32.640 a gem like this more generic outside of rails and then in rails we are going to
00:35:38.820 have glucose so the implementation is all there and grades just does you know the
00:35:43.980 integration between the configuration of rails itself and that thing
00:35:49.800 so that was it what about the range integration now we
00:35:55.079 have a gem we have to integrate with rails now the goal is we are going to ship
00:36:01.380 with two Auto loaders and you can opt out of the new one that's the plan you
00:36:06.780 can still use classic why let me tell you because for 15 years
00:36:15.660 the the previous Auto loader was brittle not his fault not its fault
00:36:23.160 it was just the way it was because of the reasons we have seen
00:36:28.260 so people had scars about this and if you if you want psychologically if
00:36:35.760 you won I I didn't want to have the experience of doing something drastic
00:36:41.280 like we are going to ship an animal to other people you are going to need to do
00:36:47.760 a migration from range 5 to range 6. because that could be a little bit scary
00:36:52.820 okay so I wanted people to feel safe in a way
00:36:58.680 so we are going to ship this this is new okay but you have the ability to migrate
00:37:06.660 raise five applications and still use the people to loader that maybe it's not perfect but that you know
00:37:13.079 and you have maybe in this tweak in this project you can this tweak in the project thought for that particular
00:37:18.119 autoloader uh peace of mind uh maybe you later you know after the
00:37:24.960 grade you can you can do like a task force or something like that that is more scope but you had a way out and this was for
00:37:33.180 me very important for users of rails now if you are starting a new brand new
00:37:39.599 application yes the default is the new one the new one is better but if you are upgrading
00:37:46.380 peace of mind you can still work with a classic auto loader and you could do that for two releases
00:37:53.280 now the tricky part is the Classic Auto loader so we have to ship to and for this reason of Peace of Mind
00:38:00.720 I want to absolutely make sure the plastic autoloader works as it did in
00:38:06.359 race 5. oddly no difference so if you start you know uh making both
00:38:14.640 coexist with if and then and you know and tweaking the logic how can you make
00:38:20.400 sure that that's going to be the case well generally you can
00:38:26.760 my idea here was I am not going to touch
00:38:32.040 one single comma of the previous Auto loader that was the way I could
00:38:38.040 guarantee that the previous Auto loader was going to work in your upgrade as it
00:38:43.079 did before despite making these changes and let me show you the technique
00:38:50.339 so when rate is boot this is code from Radio 6 and 37 this code does not exist
00:38:55.740 anymore okay so when race boat in rail 6 by the end of the boot process
00:39:01.260 there's code that says okay if you selected zyback mode which is the name of this new mod
00:39:08.220 uh this yes then then we are going to require an integration code and that iteration code
00:39:15.960 is going to take over the other Auto loader so it's like it's like we are going to hijack so and
00:39:23.940 we have no touch of one single comma of active support dependencies that will be
00:39:30.240 take over take over defines the auto loaders and then decorates dependencies let me
00:39:38.220 give you an example for instance the clear method is the one that is called when you reload okay this is the
00:39:43.800 implementation in the previous autoloader so the integration defines also a clear method
00:39:50.940 that does reloading users using the new thing and then you go and prepend this module
00:39:57.960 on the other module so we are hijacking so this this is not
00:40:04.079 calling super like you would do in a typical decoration it's not calling super this is hijacking this is I am
00:40:10.500 taking over the calls because this is public API this is another difficulty this is all public API so this public
00:40:15.839 API has to still exist and has to behave ideally as as it behaves in the other
00:40:22.079 one so this is a technique you hijack now in the module there's like I don't know
00:40:29.400 uh 30 private methods that where the implementation of the of the previous
00:40:35.940 Auto loader and they are still there in this this way okay because you only
00:40:42.720 hijack the public interface that the one that you need but those private methods
00:40:47.880 that that were needed by the previous implementation well
00:40:53.460 is that a problem no nobody is going to call them they are private and they so the entry point to those methods is cons
00:41:01.260 missing that is no longer running so that is going to be that code fine we
00:41:06.480 didn't touch anything all right this is ready but we have the
00:41:13.200 gem first version we have the integration but this is in a branch we do not merge
00:41:20.280 why because of the secret so
00:41:25.859 the last blocker the last blocker was that module autoload was deprecated
00:41:37.380 yes which level of deprecation let me show
00:41:44.160 you this is much okay I just strongly discourage the use
00:41:49.440 of autoload in any standard libraries where autoload will be that
00:41:57.420 and then in in the middle of the description says so I heard me declare
00:42:02.940 the future deprecation of Auto load like this is level 11.
00:42:11.400 all right so at this point you may ask
00:42:16.619 so you have done all these you have gone through all these
00:42:21.839 knowing that probably it is going to be for nothing because this is very very explicitly it
00:42:29.640 has been deprecated for seven years I believe it was so it was like
00:42:35.280 you know have you gone through all these
00:42:40.440 and done this work knowing that probably is not going to see the light of the day yes I did
00:42:48.599 why for the fun of it for something that I guess I believe we
00:42:54.599 all in this room sure which is the joy of problem solving this was my Grail so
00:43:01.200 just solving the problem was an enormous personal satisfaction for me
00:43:07.859 and this is out of my control so let's see all right
00:43:14.160 now there was some discussion in this threat because we said we have these plans what
00:43:21.720 do you think about that and I want you to I want to share with you this paragraph that I wrote
00:43:28.920 because this belongs to in my view open source etiquette okay
00:43:34.500 so when I entered this discussion I said first and foremost I like to be very
00:43:39.660 straightforward saying that if kernel Auto load is killed tomorrow that could be fine with me despite
00:43:46.800 everything that could be fine and I am being absolutely sincere here
00:43:53.280 I shoot down driver and really six plants Without Regrets
00:43:59.880 it's I took I took the risk is my responsibility so if you move forward with this
00:44:07.260 application you have nothing to say go ahead it's fine
00:44:12.300 the Ruby core team are the stead Wars of the language you know better than anybody else the rationale for your
00:44:19.800 decisions and if you believe you have to definitely say goodbye to Colonel out a
00:44:25.500 lot I profoundly respect it and that's part of Open Source etiquette
00:44:31.500 that when you want to interact with someone that is running a project you
00:44:36.839 have to take into account that maybe you do not have the whole context that maybe they have like a perspective of how
00:44:42.420 things fit together or how things are going on you know they may have different preferences they
00:44:48.900 may have different trade-offs than you have if you accept something you are going to
00:44:54.420 maintain it forever so uh to me being respectful for other
00:45:01.920 team members and accepting their you know decisions about ideas that you may
00:45:07.020 have for me that is fundamental now having said that yes
00:45:12.690 [Applause]
00:45:19.800 no having said that we can have a technical discussion this which is these
00:45:24.900 were the problems with the Classic Auto loader we all know that for 15 years this has been brittle and this
00:45:31.440 particular implementation solves them and also um
00:45:39.500 discussing with a working implementation it's much much much easier discussing with code that you can see
00:45:47.640 it's much much easier than discussing you know ideas you know potential Solutions
00:45:53.940 so no match enters the conversation and says okay I withdraw the propulsion
00:46:04.640 [Applause]
00:46:10.680 so when I saw this reply from match I watched it was a
00:46:15.720 speechless this for me this speaks volumes from much despite you know all the context about
00:46:22.920 that deprecation he still listens and he still is
00:46:28.140 understanding and generous enough to go and say all right let's with Rob this speaks volumes for me
00:46:36.660 so we have it all we have it we made it these ships with Reigns better too the
00:46:44.460 first thing February 25 2019
00:46:50.339 now let's take a break let's let's talk about a curiosity of the project that I
00:46:55.740 want to share with you the name zyberg so originally the gem was called it auto
00:47:02.700 loader simple autolog so I have the gem written called
00:47:10.380 it auto loader the integration in rail is written with auto the blog post written without another and then I'm
00:47:17.760 ready I push the gem and ruby gems tells me that the name is taken
00:47:23.300 [Applause] I haven't checked it
00:47:29.040 yes so there was a gem written a while back call it auto loader I couldn't reach the author what you do I had I had
00:47:36.660 to find a name quickly so I look at for inspiration in things to
00:47:41.940 see okay what is going to be the name no I I checked some options and look at
00:47:50.220 for inspiration in things that I like okay so one thing that I like very much
00:47:56.819 is watchmaking mechanical watches okay what's making has a very unique for me intersection of
00:48:05.460 engineering of craftsmanship of artistry of
00:48:11.520 acquisite attention to detail that attracts me a lot
00:48:17.520 and watch aficionados often have something that we call the Grail watch
00:48:23.940 The Grail watch is that particular there's ton of watches out there okay but but for many people there's one
00:48:31.260 that's the one that speaks to you that's the one that if things go well someday
00:48:37.200 perhaps maybe you are going to have I had a great watch
00:48:43.740 let me show you this watch
00:48:50.630 [Music]
00:49:08.700 foreign [Music]
00:49:16.650 [Music]
00:49:38.220 foreign [Music]
00:49:51.720 this was my great watch and I was kind of working on my gray
00:49:57.180 project so I connect both ideas and I said okay I'm going to connect my two grades with a
00:50:03.420 name and that was diver now
00:50:08.720 diverg is a German word I do not speak German so
00:50:16.140 Martin shurik which is a colleague in a previous client of mine kindly recorded
00:50:22.260 with beautiful boys the correct pronunciation of zyberg in German which
00:50:28.500 I would like to share with you yeah there you go
00:50:35.760 second beautiful right Martin this audio isn't is in the
00:50:42.960 Repository yeah and for complete
00:50:50.960 to to make this section complete I have to share that eventually a time later I
00:50:56.819 got the watch because you have to fulfill your dreams
00:51:04.800 okay we are in better too we need to go from this first implementation to rails six
00:51:13.500 final um how did I test the integration well I
00:51:19.619 am also the author of range contributors rage contributor is a super small application has three Auto load paths 12
00:51:25.559 classroom modules so enough enough diversity of things to get
00:51:31.319 the basics right okay but it's a tiny implication now
00:51:36.780 something amazing happened which is that from here I go to which
00:51:42.960 other application to Shopify so from this toy now Shopify wants to
00:51:50.280 integrate zyberg in the in their monolith in their application
00:51:55.619 and this is the best that could happen because we are better we are on time to polish things we are on time to make
00:52:03.000 backwards incompatible changes and because Shopify is a huge
00:52:08.760 application it has 90 enzymes six more than 600 out of maths 25k classes and
00:52:17.819 modules so if something of performance or anything
00:52:23.579 uh has to be polished is going to show up here this is what you want at this
00:52:28.740 stage this was amazing for the project now Shopify is huge we have a tentative
00:52:35.400 first non-validated implementation of something which is a little bit tricky
00:52:42.300 who is going to do this because man this is like migrating this application with
00:52:48.180 something that is tricky not validated uh you need something you need someone
00:52:53.220 that is really good and yes that was jean-vous here from Shopify
00:52:59.220 yes please go ahead he totally deserves
00:53:06.599 Jean was here so we did an amazing collaboration with John for
00:53:13.140 I don't know for a couple of months intensively and then forever because
00:53:19.380 John has been always you know in contact with cyber proposing things so we
00:53:25.260 discovered apis that were wrong we discovered performance issues or or
00:53:30.540 things that were right but you first write it right and then make the performance and so all anything that
00:53:38.040 could be improved was Amplified by running on Shopify this was
00:53:44.220 amazing the gem was way better and the race integration was great way better
00:53:50.460 thanks to this collaboration with John let me show you this tweet Jean pops up
00:53:57.720 everywhere with amazing work I know you are not on Twitter but thank you by root
00:54:03.540 which is his Handler and then the folk says Jean is most definitely the unsung
00:54:10.200 hero of the Ruby Community I love working with that guy and I personally totally subscribe this works this watch
00:54:17.760 yes [Applause]
00:54:24.059 and I am also very happy to share with you that John just joined it real score
00:54:30.180 last week yes [Applause]
00:54:39.359 also an update on that tweet uh he's on Twitter now [Laughter]
00:54:46.319 all right Shopify is completely my this migration in May 3 2000 2019 three
00:54:52.500 months three months before Race Six final so that gave me a boost of confidence
00:55:01.559 in shipping this integration with rail 6 finite because Shopify was running in
00:55:07.920 production with this and in development as well of course uh you know for three months so yeah I'm sure some you know
00:55:15.720 some details are going to show up in other projects whatever you know but man this is this is running Shopify so
00:55:21.240 that's that's a great test for what we are going to ship
00:55:26.280 and rail 6 ships with this alternative implementation in August of 2019 finally
00:55:36.059 no this is out okay and people start to do
00:55:42.119 migrations uh I want to share just a couple of them
00:55:47.280 the first one that has a special place in my heart is the migration of nanocy nanoes is a static website generator so
00:55:54.240 this is this is a gem not a range application this is a gem so nowadays
00:55:59.520 there's like two 260 gems more or less loading with zyberg which was the vision
00:56:06.920 you have not only race applications but every Gem and indeed other web fragworks
00:56:12.059 can use the autoloader so this is the entry point for Nano C
00:56:20.040 and Nano C instead of doing cherry picking requires individual files
00:56:26.420 well it maintained this list in the entry point so it's kind of bigger loading
00:56:32.460 right so at least you do not have to be doing requires manually everywhere inside the project you put it all in the
00:56:40.260 in the entry point done all right the patch of Nano C Nano C
00:56:48.440 had this patch the day after the first beta was out of diverg
00:56:56.460 and the patch looked like this everything delete yes
00:57:05.819 to me it was amazing to see these parts because it's yes yes that's that's the vision that's what we were after okay
00:57:14.940 you have to think that this these are uh this list of requires has to be
00:57:22.260 maintained so every time you add a file or remove a file you have to remember to go here and edit this list and this list
00:57:29.819 has to be ordered because if files you know if when you execute the file you
00:57:34.980 are referring to a constant at the top level because you are including a mixing or whatever well that has to be loaded
00:57:42.900 before here so it has to be maintained and in order
00:57:48.839 so the patch just deletes this deletes this this is the first gem to the best of my
00:57:54.300 knowledge that use it diver it uses Driver Way Beyond before rails and the patch watch was merged
00:58:02.520 with the first stable release of cyber the other one that they want to share
00:58:08.880 this there's many but uh I have some memories of this one this course migrated in October of 2018.
00:58:16.800 that's the that's the pull request right there so this patch migrating this course
00:58:24.660 has 500 required dependency delete
00:58:30.540 and that was for me like a trophy because required dependency is it was uh
00:58:36.660 we have not seen it but most of that infinite list of coaches
00:58:42.799 was like this is the gocha this is why it happens
00:58:48.660 and this is the solution of the workaround and the workaround oftentimes was put a required dependency for
00:58:55.020 something that you want to make sure is loaded at this point so
00:59:01.020 this was like a trophy why because you can imagine that behind every single one
00:59:07.319 of those require dependency there was a headache there was a puzzling time of
00:59:14.040 something that was not loading sometimes like a Heisenberg that sometimes happens sometimes not it happens in CI why is
00:59:21.180 not this constant not found here but in other runs it is found
00:59:26.579 you have to do research and late at some point you realize oh
00:59:31.859 we are facing this particular gocha in the list we need this required dependency I am sure each single one of
00:59:40.380 those required dependency was a pain in the ass so getting rid of this is yes that was
00:59:47.160 it so if you are migrating uh to to enjoy
00:59:54.540 work there's a guide that is going to help you with everything you have you
00:59:59.940 want to know about doing this and finally
01:00:07.200 uh finally after all this journey something
01:00:12.839 really beautiful happened this year which is that I got a Fukuoka groupie award for this project
01:00:19.150 [Applause]
01:00:27.839 this was an incredible recognition for this work and it made me be proud and
01:00:33.420 happy but there's there's something that still makes me happier which is that
01:00:41.660 issues related to auto loading have mutually banished so before these it was
01:00:47.760 a constant stream of issues and tickets that it not because of the tickets
01:00:53.160 themselves but because that means people are having problems with framework with the
01:01:00.420 framework within track structure instead of focusing on the work on the applications
01:01:06.720 that was not a good user experience at all so use so issues have beautifully
01:01:14.099 banished they are gone it just works so I monitor the rails repository I monitor
01:01:21.540 Twitter if you want to reach me on Twitter you you mentioned cyborg I'm going to see it
01:01:26.819 uh stack overflow it just works
01:01:33.180 so zyberg got the spotlight in this particular presentation
01:01:40.079 but the goal of the project is to banish the goal of the project is that it works
01:01:46.040 so smoothly so transparently that you indeed forget that is even doing just
01:01:52.619 discretely his work and that's it all okay thank you
01:01:57.870 [Applause]